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ABSTRACT 

The  purpose  of  this  thesis  is  to  provide  information  about 
wargaming  to  prospective  users.  An  in  depth  discussion  focusing  on 
two  primary  components  of  wargaming  is  presented.  The  first 
component  is  wargaming  in  general.  A  description  is  provided  of 
wargaming  features  and  a  discussion  concerning  the  limitations  and 
hazards  inherent  to  wargames.  The  second  component  is  how  to 
design  a  wargame  experiment  to  support  command,  control  and 
communications  (C3)  hypotheses  testing.  A  description  of  the  steps 
in  planning  a  general  wargame  experiment  including  detailed 
information  on  wargame  software  considerations  is  presented. 
Finally  a  synopsis  followed  by  a  critique  of  an  actual  C3  wargame 
experiment  is  provided  as  an  example  of  the  principles  given  in 
this  thesis. 
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I.  INTRODUCTION 

A.   BACKGROUND 

Peacetime  operations  that  can  be  observed  stimulate  little 

of  the  activity  that  can  be  expected  in  crises  or  wartime,  and 

fleet  exercises  that  offer  an  opportunity  to  observe  combat 

direction  offer  little  opportunity  to  observe  higher  level 

command  functions  in  a  realistically  stimulated  environment 

(Ketron,  Inc.  1979,  p.  4).  Before  Desert  Storm,  many  weapons, 

systems,  and  ideas  were  untested  in  combat.   The  Navy  cannot 

rely   on   actual   conflict   for   evaluating   its   combat 

capabilities.     Wargaming  used   as   a   tool   in   research, 

operational  planning,  and  training  is  rapidly  expanding  to 

meet  these  needs. 

Improvements  will  be  needed  to  adapt  existing  systems  and 
technology  and  to  make  limited  upgrades  and  changes  in  a 
number  of  areas,  including,  the  following: .. .Simulation- 
gaming  support  to  assist  in  near  real-time  analysis  of 
alternative  courses  of  action  and  for  peacetime  training 
and  readiness....  (Command  and  Control  Functional 
Analysis  and  Consolidation  Review  Panel  Report,  1991,  p. 
7) 

Interest  in  improving  the  Navy  command,  control  and 

communication   (C3)   systems   has   been  encouraged  by   the 

Department  of  Defense  in  recent  years  for  several  reasons. 

The   threat   to   Naval   forces   continues   to   become   more 

complicated  due   to  complex,   multi-dimensional   anti-ship 

missiles  deployed  on  aircraft,  ships,  and  submarines,  and 


increasingly  sophisticated  electronic  warfare  technology. 

There  has  been  an  exponential  growth  in  the  amount  of 

information   available   to   commanders   for   assessing   the 

situation  and  operational  planning.    Rapid  advances  in 

computer  and  communications  technologies  have  enabled  the 

planning  and  design  of  increasingly  sophisticated  systems  to 

support  tactical  C3. 

Modeling,  simulation  and  gaming  can  effectively  be  used 

for   identifying  requirements   and  designing,   developing, 

acquiring,   training  for  and  employing  major  C3  systems. 

Command   and  control   of   military   forces   and   assets   is 

ultimately  a  human  decision  process  supported  by  C3  systems 

with  human  operators  receiving  imperfect  information.   The 

most   effective   and   economical   way   to   evaluate   systems 

involving   human   processes   is   by   using  wargaming.     In 

particular,  wargaming  best  supports  research  for  the  following 

command  and  control  functions:  situation  monitoring,  decision 

making,  and  management  of  forces  and  intelligence  support 

assets. 

The  first  step  in  command  and  control  research  (C2R) 
focuses  on  decisions  critical  to  military  operations,  the 
two  most  important  being  to  identify  decisions  that  have 
an  impact  on  combat  outcome;  and  to  identify  the 
appropriate  command  levels  where  different  decisions  are 
most  appropriate.  Once  the  crucial  decisions  are 
identified,  C2R  then  encompasses  many  issues,  including 
identifying  information  required  to  make  the  best 
decisions  and  the  effect  of  time  on  the  information 
gathering  and  dissemination  process.  (Hogan  and  Might, 
1988,  p.  33) 


Navy  officers  need  to  be  aware  of  the  benefits  and  hazards 

of  using  wargaming  for  research,  operational  planning  and 

training.   Decision  makers,  who  have  not  been  involved  in 

wargaming,  can  expect  to  deal  with  games  or  the  results  of 

games  in  the  course  of  their  careers.   Wargame  use  is  as 

complex  as  the  situations  they  attempt  to  address. 

The  fact  is  that  wargames  and  wargaming  are  consistently 
misunderstood,  denigrated,  even  denounced,  not  only  by 
gaming  outsiders,  but  also  by  gaming  proponents  and 
practitioners.  This  unfortunate  situation  is  a  result  of 
a  failure  to  reconcile  the  fundamental  ambiguities  of 
wargaming,  to  understand  the  nature  of  the  tool  -  the  game 
-  and  of  the  process  of  using  it  -  gaming.  (Perla,  1990, 
p.  2) 

In  addition  to  limitations  inherent  in  the  games,  game 

data  users  need  to  question  and  understand  the  process  of 

applying  a  wargame  to  an  experiment.   An  example  of  wargame 

abuse  is  the  wargame  that  was  conducted  in  the  spring  of  1942 

at  the  War  College  in  Tokyo.   Japanese  naval  planners  were 

preparing  plans  for  the  capture  of  Midway  and  the  western 

Aleutians  in  early  June,  the  seizure  of  strategic  points  in 

New  Caledonia  and  the  Fiji  Islands  in  July,  air  strikes  on 

southeast  Australia,  and  operations  against  Johnston  Island 

and  Hawaii  in  August. 

During  the  play  the  Nagumo  Force  was  attacked  by  land- 
based  air  while  its  own  planes  were  attacking  Midway. 
Following  the  rules  of  the  game,  an  umpire  determined 
that  the  carriers  received  nine  hits  and  that  two  of 
them,  the  Akagi  and  Kaga,  were  sunk.  Rear  Admiral 
Ugaki,  the  director  of  the  game,  arbitrarily  reduced 
the  number  of  hits  to  three,  and  the  number  of 
sinkings  to  one,  and  then  permitted  the  sunken  carrier 
to  participate  in  the  next  part  of  the  play  dealing 


with  the  New  Caledonia  and  Fiji  Island  invasions. 
(McHugh,  1966,  p.  2-19) 

Wargaming   and  model   use  will   continue   to   grow   in 

importance  as  funding  is  reduced  and  the  need  for  analysis 

tools  increases.  Wargame  adaptability  uniquely  applies  to  the 

ever  expanding  and  diverse  third-world  military  scenarios,  the 

continuous  redrawing  of  global  country  boundaries,  and  unique 

combinations  of  our  own  forces. 

B.   PURPOSE  AND  GOALS 

The  principal  purpose  of  this  thesis  is  to  provide 
information  about  wargaming  to  prospective  users.  The  first 
component  is  wargaming  in  general.  The  goal  is  to  provide  the 
reader  with  limitations  that  need  to  be  considered  and  hazards 
that  must  be  avoided  when  using  conclusions  derived  from 
wargame  data.  The  thesis  contains  a  description  of  wargame 
features  and  a  discussion  concerning  wargame  limitations  and 
hazards.  The  second  component  is  how  to  design  a  wargame 
experiment  to  support  C3  hypotheses  testing.  The  goal  is  to 
enable  the  reader  to  test  C3  hypotheses  using  a  wargame 
experiment  by  providing  a  guide.  A  general  description  of  the 
steps  in  planning  a  general  wargame  experiment  is  provided, 
including  detailed  information  on  wargame  software 
considerations.  A  synopsis  followed  by  a  critique  of  an 
actual  C3  wargame  experiment  is  furnished  apply  the  principles 
supported  in  this  thesis. 


C.   ORGANIZATION  AND  SCOPE 

This  thesis  is  an  implementation  guide.  Chapter  I 
provides  the  reader  with  a  background  on  why  wargaming  is 
advantageous  for  research  and  will  be  even  more  useful  in  the 
future.  It  also  includes  the  thesis  purpose,  goals,  and 
organization.  Chapter  II  goes  into  more  detail  on  C3  wargame 
applications,  types,  features  and  basic  requirements.  Chapter 
III  describes  the  limitations  inherent  to  wargaming  and 
hazards  to  avoid  in  wargame  use.  Chapter  IV  guides  the  reader 
through  the  steps  for  a  general  experiment  design,  including 
detailed  information  on  the  critical  stage  of  software 
selection.  Chapter  V  provides  a  synopsis  of  an  actual  C3 
experiment  that  used  a  wargame  to  evaluate  a  decision  aid. 
Then  the  example  is  critiqued  to  apply  the  principles  given  in 
this  thesis.   The  final  chapter  is  a  summary. 


II.   WARGAMING  DESCRIPTION 

Before  going  into  the  specifics  of  wargaming 
considerations  and  experiment  design,  this  chapter  ensures 
that  the  reader  understands  the  author's  idea  of  what  a 
wargame  is,  some  general  and  C3  wargame  applications,  the 
types  of  wargames,  and  common  features  and  basic  requirements 
of  a  wargame. 

A.   DEFINITIONS 

In  order  to  begin  with  a  common  understanding  of  terms 
used  in  this  thesis,  the  following  definitions  apply. 


•  Models  are  simplified  approximations  of  reality.  A  model 
is  a  representation  of  an  object  or  structure,  or  an 
explanation  or  description  of  a  system,  a  process,  or  a 
series  of  related  events  (McHugh,  1966,  p.  1-4).  The 
amount  of  detail  and  variables  in  a  model  can  vary  to  fit 
its  use.  Some  models  look  like  what  they  represent,  some 
are  similar  and  some  are  symbolic. 

•  A  wargame  is  a  theoretical  representation,  in  accordance 
with  predetermined  rules,  data,  and  procedures,  of 
selected  aspects  of  a  conflict  situation  or  military 
operation.  A  gaming  exercise  employs  human  beings  acting 
as  themselves  or  playing  simulated  roles  in  an  environment 
which  is  either  actual  or  simulated  (PACER  Systems,  Inc, 
1975,  p.  11) . 

•  A  simulation  is  an  operating  representation  of  selected 
features  of  real-world  or  hypothetical  events  and 
processes  (McHugh,  1966,  p.  1-1).  All-machine  simulations 
(no  human  decision  making)  are  not  defined  as  games.  All 
games  however,  are  simulations. 


There  is  a  distinct  difference  between  a  simulation  and  a 

wargame  that  needs  to  be  understood  for  C2  applications. 

In  simulations,  explicit  rules  and  procedures  within  the 
simulation  help  define  available  choices  and  decisions. 
Conversely,  in  war  games,  players  interactively  make  the 
pivotal  decisions  and  choices.  In  this  context,  war  games 
are  simulations  with  participatory  user  interaction. 
(Airpower  Research  Institute,  1985,  p.  3) 

Use   of   strictly   simulation   is   not   recommended   for  C2 

hypothesis  testing  experiments  because  human  impacts  and 

dimensions  are  not  easily  duplicated  by  a  computer  program. 

In  model  building,  C3I  is  generally  recognized  as  one  of 
the  most  difficult  areas  to  portray.  Within  the  open 
simulation  (man-in-the-loop)  framework,  some  of  the  burden 
is  removed  from  the  model  designer  when  the  decision  is 
made  to  allow  the  player  to  perform  the  command  and 
control  (C2)  functions  directly.  The  remaining  processes, 
intelligence  and  communications,  are  somewhat  more 
mechanical  and,  therefore,  simpler  to  represent  in  a 
rational  fashion.  (National  Defense  University,  1986, 
Volume  I,  p.  IX- 1) 

In  addition  to  understanding  what  wargaming  is,  it  is  also 

useful  to  understand  what  wargaming  is  not.   Wargaming  is  not 

a  technique  for  producing  a  rigorous,  quantitative  or  logical 

dissection  of  a  problem  or  for  defining  precise  measures  of 

effectiveness  by  which  to  compare  alternative  solutions 

(Perla,  1990,  p.  164).   Wargaming  is  not  real.   The  level  of 

abstraction  depends  on  the  model  and  the  application.   A 

wargame  is  not  duplicable.   The  interplay  of  human  decisions 

affects  the  simulated  outcomes  and  therefore,  does  not  allow 

the  researcher  to  expect  the  same  sequence  of  decisions  and 

outcomes  during  a  replay  of  a  game.   Wargame  designers  and 

analysts  involved  in  research  need  to  be  careful  to  define  the 


results  of  a  wargame  in  terms  of  why  and  issues  raised,  not 
what  happened.  A  game  should  not  be  used  to  predict  outcomes. 
They  are  best  used  to  investigate  processes. 

B.   GENERAL  WARGAME  APPLICATIONS 

This  section  discusses  why  professional  wargames  are 
played  in  addition  to  its  traditional  use  for  training.  Some 
of  the  added  applications  for  wargames  are  to  investigate  not 
rigorously  quantifiable  problems,  decompose  complex  problems, 
uncover  new  aspects  of  a  problem,  test  operational  plans, 
assist  in  the  planning  process,  and  focus  on  human  behavior. 

1.  Explore  Not  Rigorously  Quantifiable  Problems 

For  this  type  problem,  the  choice  of  which  variables 
form  the  basis  of  a  problem  and  the  relationships  of  the 
variables  are  not  easily  determined.  An  example  of  a  not 
rigorously  quantifiable  problem  is  determining  the  United 
States  strategic  forces  necessary  to  deter  nuclear  war.  A 
rigorously  quantifiable  problem  would  be  determining  the 
impact  point  of  a  bomb  dropped  by  an  aircraft.  Selecting  an 
output  measure,  determining  relevant  variables,  and  defining 
the  underlying  model  is  much  more  difficult  in  solving  a  not 
rigorously  quantifiable  problem.  Wargames  represent  the  only 
formal  decision  aid  available  for  this  type  of  problem. 

2 .  Decompose  Complex  Problems 

Another  application  is  to  use  a  wargame  as  a  tool  for 
decomposing  complex  problems.  Proper  game  preparation  forces 


a  consideration  of  the  range  of  possibilities  for  any 
assumption.  This  step  provides  a  natural  method  for 
subdividing  a  problem  into  smaller  parts.  With  wargaming 
experiments,  one  assumption  at  a  time  can  be  manipulated  to 
examine  its  impact  on  the  problem.  The  decomposition  of  a 
complex  problem  into  separate  models,  each  of  which  can  be 
evaluated  and  the  results  later  combined,  is  an  aid  to 
exploring  a  complex  problem.  However,  caution  should  be 
applied  when  making  the  final  combination. 

3 .  Uncover  New  Aspects 

Wargaming  can  be  a  form  of  brainstorming  to  generate 
aspects  of  a  situation  that  are  not  readily  apparent.  By  role 
playing  in  a  realistic  game,  participants  involved  in  a 
complex  scenario  can  explore  alternative  policies,  discover 
unexpected  alternatives,  and  sometimes,  anticipate  outcomes 
that  differ  from  those  originally  envisioned  (Airpower 
Research  Institute,  1985,  p.  44).  This  approach  can  be  used 
in  research  to  generate  and  test  hypothesis  concerning  human 
interactions. 

4.  Test  Operational  Plans 

Another  use  of  wargaming  is  testing  operational  plans. 
The  experiment  requires  a  realistic  model  in  order  to  apply  a 
real-world  situation  that  requires  planning  a  complex  sequence 
of  actions.  Insights  may  be  gained  as  to  how  the  plan  will 
work  in  the  real  world  by  playing  out  the  planned  actions.   If 


the  initial  design  of  a  game  incorporates  well-known  critical 
factors  into  its  models  and  procedures,  the  play  of  the  game 
and  the  questions  and  issues  it  raises  can  lead  to  the 
discovery  of  other  factors  whose  importance  may  have  been 
previously  unsuspected  or  undervalued  (Perla,  1990,  p.  181). 
The  reader  is  reminded  that  insights  not  predictions  of 
possible  outcomes  will  be  generated. 

5.  Assist  in  Planning  Process 

Since  a  wargame  can  be  used  to  evaluate  a  plan,  it  can 
also  be  used  to  integrate  plan  evaluation  into  the  planning 
process.  First  a  plan  is  gamed  and  evaluated,  then  any 
deficiencies  can  be  corrected  in  the  next  iteration.  The  idea 
is  to  create  a  planning,  testing,  and  replanning  cycle.  The 
more  complex  the  situation  and  therefore  the  plan,  the  more 
useful  and  practical  is  the  wargaming  application.  The  design 
of  the  game  (organizing)  ,  and  the  play  and  subsequent  analysis 
of  the  game  (exploring)  form  a  circular  chain  or  feedback 
loop,  in  which  the  questions  and  issues  arising  from  one  play 
of  the  game  can  reshape  or  reorganize  the  game  system  itself 
to  make  it  a  more  accurate  representation  of  reality  (Perla, 
1990,  p.  180). 

6.  Focus  on  Human  Behavior 

Wargames  allow  the  researcher  to  focus  on  human 
behavior,  specifically  on  human  decision  making.  The  models 
usually   offer  various   outcomes   based   on   a   probability 
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distribution.  Thus,  the  "roll  of  the  dice"  provides  a  wide 
range  of  possible  results  or  snapshots  of  reality  with  which 
the  players  must  deal  (Perla,  1990,  p.  276).  The  value  of 
wargaming  for  decision  making  is  in  qualitative  assessments  of 
why  decisions  are  made.  The  advantage  of  a  wargame  over  an 
exercise  is  that  in  an  exercise  there  are  limits  on  the  range 
of  physical  parameters  and  processes  and  on  the  range  of 
potential  decision  choices  participants  can  choose  from. 

C.   C2  WARGAME  APPLICATIONS 

In  model  building,  C3  is  recognized  as  one  of  the  most 
difficult  areas  to  represent.  Because  wargaming  uses  a  man- 
in-the-loop  framework,  some  of  the  complexity  in  model 
building  is  reduced  by  allowing  the  player  to  perform  the 
command  and  control  (C2)  functions  directly.  The 
communications  function  is  more  mechanical  and  can  be  simpler 
to  represent  in  the  model. 

There  are  at  least  four  aspects  of  command  and  control 
activities  and  processes  that  can  be  approximated  in  wargames 
because  they  are  characteristics  that  can  be  represented 
realistically  enough  in  the  games  to  support  meaningful 
analysis.   These  aspects  are: 


•  Command  roles.  Decision  making  functions  and  the  nature 
and  scope  of  the  force  actions  that  result  from  the 
decision  making  responsibility  are  defined  by  doctrine  and 
convention  so  there  should  be  very  little  discrepancy 
between  command  roles  in  reality  and  those  for  a  game. 
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•  Hypothesis  formation.  Any  time  that  decisions  must  be 
made  in  the  face  of  uncertainty  and  incomplete 
information,  the  decision  maker  must  construct  from 
available  information  hypotheses  as  to  the  nature  of 
events,  situations,  etc.,  about  which  he  has  no  knowledge 
and  make  decisions  as  if  those  hypotheses  were  true 
(Ketron,  Inc.,  1979,  p.  26).  This  holds  true  for  gaming 
if  the  decision  makers  are  playing  a  similar  game  role  as 
the  role  they  hold  in  reality. 

•  Filtering  and  interpretation  of  information.  The  way  a 
commander  identifies  items  relevant  to  a  decision  by 
filtering  the  constant  stream  of  information  against  his 
hypothesis  to  arrive  at  a  decision  is  realistic  in  a 
wargame  if  the  actual  information  that  the  decision  maker 
is  filtering  and  interpreting  is  realistic  in  format, 
content,  and  order  of  arrival. 

•  Information  categories.  Given  that  the  information  that 
is  provided  is  tactically  realistic,  and  that  the  roles 
and  reasoning  just  discussed  reflect  those  which  would  be 
encountered  in  the  real  world,  the  requests  for 
information  to  support  command  decision  making  should  also 
be  representative  of  real  world  operations  (Ketron,  Inc., 
1979,  p.  27)  . 


These  aspects  focus  on  the  exercise  of  decision  making  along 
the  chain  of  command  and  are  more  dependent  on  the  human 
element  of  C3. 

D.   TYPES  OF  WARGAMES 

Wargames  can  be  classified  in  many  and  multiple  ways.  The 
following  gives  a  definition  for  classification  by  technique 
of  rule  application,  wargame  purpose,  and  geographic  scope  and 
decision  making  level. 
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1.   Classified  by  Technique 

The  two  techniques  of  applying  rules  are  rigid  games 
and  free  games.  A  technique  choice  for  knowledge  about  the 
opponent's  forces  is  open  or  closed  games. 

Rigid  games  are  conducted  by  following  detailed, 
nondeviating  rules.  The  game  rules  usually  provide  for  chance 
events;  the  role  and  the  range  of  chance  results  are  not  left 
to  the  judgment  of  an  umpire  or  controller  (Hausrath,  1971, 
pg.  123).  All  possible  outcomes  are  predetermined  and  rules 
provide  for  one  possible  outcome  for  each  possible  result  of 
chance.  For  example,  if  there  is  a  50:50  chance  of  either  of 
two  outcomes,  the  rules  determine  how  the  result  will  be 
resolved  such  as  a  coin  toss.  Rigid  games  have  extensive 
rules  and  are  more  complex  to  write  for  computer  simulation. 
Rules  provide  for  identical  procedures,  methods,  and  source 
data  in  each  run  and  therefore  are  believed  to  provide  more 
consistent  and  reliable  data. 

A  free  game  is  conducted  with  fewer  fixed  rules  and 
the  assessments  are  made  by  an  umpire  or  controller.  The 
controller  bases  judgments  on  military  experience.  It  is  less 
tied  to  the  mechanics  of  play  and  is  therefore,  less  reliable 
than  a  rigid  game  and  is  usually  more  applicable  to  training 
use. 

The  open  game  is  simple  and  informal.  The  opposing 
players  have  full  knowledge  of  the  positions  and  actions  of 
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their  own  and  their  enemy's  forces  so  there  is  no  need  for  an 
intelligence  model.  The  open  game  is  most  often  used  for 
training. 

The  closed  game  is  more  complex  and  formal.  Each  team 
has  first  hand  knowledge  only  of  their  own  forces  and  partial 
knowledge  of  the  opposing  team  is  received  by  an  intelligence 
model.  The  closed  game  more  closely  approximates  reality  and 
is  more  useful  in  command  and  control  hypothesis  testing. 

2.  Classified  by  Purpose 

When  the  primary  purpose  of  a  wargame  is  to  provide 
the  players  with  decision  making  experience,  the  game  is 
educational.  When  a  game  is  conducted  in  an  attempt  to  obtain 
information  and  data  that  will  help  the  responsible  commander 
to  make  decisions,  the  game  is  an  operational  planning  type. 
If  the  purpose  is  hypothesis  testing,  the  game  is  an 
experiment  for  research. 

3.  Classified  by  Level 

This  categorization  combines  geographic  scope  and  the 
level  of  decision  making.  The  three  levels  are  : 
global/strategic,  theater /operational,  and  local/tactical. 

In  global/strategic  games,  the  participants  are  top 
level  decision  makers,  and  the  goal  is  to  improve  their 
perspective,  test  strategies,  and  identify  important  issues  at 
that  level.  The  primary  output  of  these  high-level  games  is 
qualitative,  consisting  typically  of  game  narratives  with  some 
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interpretations  of  events  and  little  numerical  data  (Perla, 
1990,  p.  170)  . 

Commanders  in  chief  of  the  unified  or  specified 
commands  are  the  primary  decision  makers  for  the 
theater/operational  level  wargame.  These  games  are  designed 
to  explore  specific  issues  and  identify  strategic, 
operational,  and  tactical  problems  with  the  theater  scope. 
The  output  is  usually  similar  to  that  obtained  for 
global/strategic  games  however,  theater/operational  games  more 
often  run  the  game  more  than  once  with  the  same  players  to 
generate  more  numerical  data. 

The  primary  decision  makers  in  the  local/tactical 
category  are  usually  battle-group  commanders  or  below.  These 
games  focus  attention  on  force  levels  and  tactical 
deployments,  weapon  and  sensor  performance,  and 
interrelationships  among  various  warfare  areas.  The  outputs 
are  a  combination  of  qualitative  and  quantitative  with  more 
numerical  data  than  the  other  game  categories.  The  number  of 
times  the  game  is  run  with  the  same  players  tends  to  be 
higher. 

E.   COMMON  FEATURES  OF  WARGAMES 

The  following  basic  characteristics  are  present  in  all 
war games. 

•  Simulates  a  military  operation. 
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•  Is  usually  at  least  a  two-sided  representation  of  forces 
opposed  to  each  other. 

•  Is  conducted  in  accordance  with  data,  rules,  and 
procedures  that  conform  to  accepted  military  doctrine. 

•  Represents  an  actual  or  predicted  real-life  situation. 

•  Provides  for  the  employment  and  tactical  movements  of 
units  in  each  force. 

•  Uses  some  graphic  means  of  identifying  units,  weapons,  and 
positions  of  forces. 

•  Provide  a  system  for  taking  into  account  the  firepower  and 
other  capabilities  of  the  forces  and  equipment  involved 
and  a  means  for  assessing  the  effects  of  combat. 

•  Includes  a  clash  of  opposing  forces  in  fire  fights, 
battles,  or  prolonged  campaigns.  (Hausrath,  1971,  p.  11) 
and  (McHugh,  1966,  p.  1-12) 


A  wargame  played  for  logistic  requirement  determination  would 
have  exceptions  to  the  above  list  of  characteristics. 

F.   BASIC  REQUIREMENTS  FOR  WARGAMES 

The  following  is  a  list  of  the  basic  requirements  that  are 
common  to  all  wargame  applications.  These  requirements  need 
to  be  determined  early  and  include:  personnel;  facilities  and 
equipment;  rules,  data  and  procedures;  and  a  scenario. 

1.   Personnel 

Because  wargaming  involves  humans  in  the  decision 
process,  an  especially  important  requirement  is  knowledgeable 
personnel.  For  C3  experiments,  it  is  critical  that  the 
players  be  highly  qualified  and  be  assigned  to  positions 
appropriate  to  their  experience  and  training.    The  game 
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designers  and  analysts  should  be  militarily  knowledgeable  and 

have  the  required  mathematical,   statistical,  and  related 

technological  knowledge  and  skills. 

These  scientists  must  translate  military  characteristics 
and  actions  into  models  and  data  that  may  be  recorded  and 
tabulated.  They  must  then  compute  the  interactions.  They 
must  assure  that  chance  events  and  results  are  treated  in 
a  sound  mathematical  and  statistical  manner  to  represent 
realistic  probabilities.  In  supervising  the  recording  and 
processing  of  game  data,  and  in  the  interpretation  of 
results,  proper  allowance  must  be  made  for  the  effect  of 
assumptions,  approximations,  and  change  results.  All 
elements  must  be  managed  within  the  limits  of  reliability 
imposed  by  the  original  input  data  and  data-processing 
methods.   (Hausrath,  1971,  p.  84) 

Most  games  currently  in  use  are  played  on  computers.   Some 

members  of  the  team  of  designers  and  analysts  must  be  skilled 

in  designing  and  programming,  modifying,  and  troubleshooting 

the  models  processed  in  the  computer;  the  supervision  of  the 

computer  input  preparation;  and  the  interpretation  of  computer 

outputs. 

2.  Facilities  and  Equipment 

The  complexity  of  gaming  facilities  and  equipment  has 
grown  with  technology.  In  the  1800 's  games  were  played  with 
physical  pieces  representing  equipment  and  units,  now  games 
are  usually  played  on  interacting  computers  with  display 
monitors.  Facilities  usually  consist  of  two  or  three 
adjoining  rooms,  each  having  access  to  a  connecting  corridor. 

3.  Standard  Rules,  Data,  and  Procedures 

Standard  rules,  data,  and  procedures  are  provided  in 
the  form  of  a  game  manual  or  handbook.     Also   a   set   of 
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reference  data  such  as  the  weapon  capabilities  and  the 

quantities  of  each  weapon  available  on  each  unit  is  required. 

4.   Scenario 

A  statement  of  the  situation  is  required  for  the 

initiating  and  conducting  of  any  game.   It  may  be  a  briefing 

and/or  a  detailed  game  directive.    The  statement  of  the 

situation  must  include  all  the  "givens"  for  the  game. 

These  "givens"  include  a  statement  of  the  geographic 
locale,  the  forces  and  weapons  available  to  each  side,  the 
time  frame  in  which  the  action  is  set,  and  the  political 
and  other  constraints  applicable  to  the  game.  Also, 
"givens"  customarily  include  a  resume  of  the  conditions 
leading  to  and  resulting  in  the  initiation  of  conflict, 
the  starting  deployment  of  forces  involved,  and,  above 
all,  orders  or  instructions  to  each  commander  stating  the 
purpose  and  objectives  of  the  game,  his  special  mission, 
and  related  details.   (Hausrath,  1971,  p.  84) 

This  chapter  has  defined  the  difference  between  a 

simulation  and  a  wargame,  wargame  applications  other  than 

training  and  logistics,  and  some  common  features  and  basic 

requirements  of  all  wargames.   The  next  chapter  addresses  the 

limitations   and   hazards   inherent   in  wargaming   use   and 

conclusion  application. 
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III.   WARGAMING  CONSIDERATIONS 

Wargames  and  the  conclusions  derived  from  wargaming  are 
often  misunderstood,  criticized,  and  even  opposed.  This 
circumstance  has  resulted  from  a  misunderstanding  concerning 
the  nature  of  wargaming.  When  its  limitations  are  understood 
and  hazards  are  avoided,  wargaming  is  a  very  valuable  tool. 

A.   LIMITATIONS 

It  is  critical  for  the  wargame  researcher  and  data  user  to 
fully  comprehend  the  limitations  inherent  to  any  wargame. 
These  handicaps  can  only  be  mitigated  and  not  negated. 
However  once  their  effects  are  understood,  wargaming  is  very 
useful  for  valid  experimental  testing. 

1.   Reality  vs.  Complexity 

Wargame  designers  tackle  a  built  in  conflict  when 
trying  to  balance  realism  versus  playability  (detail  versus 
simplicity) .  The  difficulty  is  in  designing  symmetry  between 
the  game  system  and  the  game  player.  If  the  game  system 
becomes  too  complex  when  the  designer  attempts  to  accurately 
duplicate  realism,  the  player's  behavior  may  become  unnatural. 
A  complicated  game  system  can  artificially  slow  the  players 
reactions  to  the  game  and  make  them  more  concerned  with  the 
mechanics  of  the  game.  However,  over  simplistic  game  systems 
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do  not  challenge  the  players  and  will  not  encourage  decisions 

more  like  those  made  in  a  real  situation. 

The  key  to  realistic  wargaming  lies  in  balancing  the 
player's  experience  in  his  decision  making  role  with 
as  accurate  a  representation  as  possible  of  the 
physical  outcomes  of  his  own  decisions,  his  opponents 
decisions,  and  the  objective  dynamics  of  combat. 
Achieving  this  balance  is  difficult  because  realistic 
decision  making  requires  giving  the  player  realistic 
information  and  accurately  representing  the  realistic 
effects  of  time.   (Perla,  1990,  p.  303) 

2.   Artificialities 

The  fog  of  war  cannot  be  fully  duplicated;  realistic 
information  is  difficult  to  accurately  represent  in  a  wargame. 
Typically  the  players  have  too  much  information  and  of  too 
high  quality.  Realistic  provision  for  chaos,  unanticipated 
successes  and  failures,  "someone  who  doesn't  get  the  word"  and 
a  commander  not  knowing  what  is  happening  elsewhere  in  the 
battle  are  difficult  to  represent  in  a  wargame. 

Communication  circuits  as  represented  in  a  wargame 
usually  perform  too  well.  Message  text  and  voice 
intelligibility  are  always  good.  There  are  no  communication 
outages,  and  transmissions  and  delivery  are  much  faster  than 
in  reality. 

Combat  results  can  be  modified  by  the  game  controllers 
to  continue  the  game.  An  example  would  be  to  not  allow  all 
Blue  carriers  and  flagships  to  be  eliminated  by  the  first  Red 
strike,  so  Blue  can  continue  to  play.   If  this  happens,  the 
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combat  results  cannot  be  used  to  assess  strategy  and  tactics 
used  by  either  side. 

During  the  game  play,  the  participants  can  become 
intensely  involved  but  because  the  game  is  limited  to  work  day 
hours  with  breaks  for  lunch,  the  combat  stress  is  reduced. 
Also  when  the  game  breaks  occur,  the  participants  have  an 
opportunity  to  review  their  situation.  This  assessment  time 
would  probably  not  be  available  in  a  similar  real  world 
situation. 

Each  player  approaches  the  game  with  a  preconceived 
mental  model  of  the  world  and  will  act  based  on  their  previous 
experience  and  knowledge.  This  is  called  framing.  The 
information  presented  during  the  game  is  processed  by  this 
internal  model.  Because  it  contains  a  set  of  filters  and 
sorting  and  weighing  factors,  not  all  the  information  is 
accepted.  When  a  player  makes  a  decision,  the  alternatives 
considered  fit  within  their  mental  model.  Therefore,  the 
solutions  for  the  player  to  choose  from  have  been  internally 
limited. 

Combat,  fear,  danger,  losses  and  the  unexpected  are 
missing  from  a  wargame.  Morale,  leadership,  fatigue,  and 
courage  can  be  added  to  the  list  of  factors  that  have  an 
important  effect  on  the  outcome  of  a  situation,  but  which 
cannot  be  predicted  or  measured.  An  example  is  whether  a  unit 
will  stand  and  fight  or  break.  The  will  to  fight  is  not 
quantifiable  or  even  understood. 
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Admiral  Arleigh  A.  Burke  put  his  finger  on  the  central 
artificiality  of  wargaming  when  he  said  that  "nobody  can 
actually  duplicate  the  strain  that  a  commander  is  under  in 
making  a  decision  during  combat."  In  a  wargame,  real 
forces  do  not  deploy,  real  weapons  do  not  engage,  and  real 
people  do  not  die   (Perla,  1990,  p.  250). 

These  elements  are  usually  handled  by  assuming  parity  or 

equality  for  opposing  forces. 

An  additional  artificiality  is  the  assumed  homogeneity 
of  similar  forces  such  as  all  pilots  are  equally  skilled  and 
all  guns  and  missiles  of  the  same  type  work  equally 
effectively.  These  simplifications  are  necessary  for  game 
playability. 

The  certainty  that  at  some  point  the  confrontation 
will  escalate  into  warfare  is  a  built  in  unnatural  condition 
that  affects  the  players'  perception  of  the  situation  and 
their  decisions.  In  a  real  world  confrontation,  the  option  of 
a  diplomatic  solution  exists.  As  a  result,  the  game 
commanders  are  less  troubled  by  the  possibility  that  their 
actions  will  precipitate  a  conflict  that  otherwise  would  have 
been  avoidable,  and  tend  to  be  more  aggressive  than  they  might 
be  in  a  real  world  confrontation  (Ketron,  Inc.,  1979,  p.  23). 
3.   Data  Validation 

Data  validation  is  a  measure  of  how  much  confidence 
there  is  in  the  data  for  a  given  model.  Many  times  for 
complex  and  difficult  games,  some  of  the  real  world  data  is 
not  available.  An  example  is  a  wargame  concerning  the 
benefits  of  using  a  new  system  in  combat  when  it  has  not  been 
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implemented  or  tested  in  the  real  world.  Also  the  data  can  be 
unavailable  because  the  system  for  the  model  has  not  yet  been 
built.  This  is  a  "catch  22"  predicament  because  a  lack  of 
real  world  data  for  complex  and  difficult  situations  can  be 
precisely  why  they  are  wargamed. 
4.   Workable  Falsehoods 

To  create  a  functioning  wargame,  the  designer  must 
apply  some  basic  assumptions.  In  reality  these  assumptions 
are  incorrect,  but  they  are  minor  falsehoods  that  the  designer 
and  researcher  can  manage  to  deal  with. 

a.  The  Right   Variables  Have  Been  Selected 

That  the  right  variables  have  been  selected  is  the 
first  assumption.  A  close  examination  of  the  hypothesis  and 
issues  will  identify  a  large  number  of  variables.  A  good 
workable  model  will  not  be  able  to  address  all  of  the 
variables.  It  will  only  be  able  to  deal  with  the  set  of 
variables  that  are  high  priority  and  only  a  few  if  any  of  the 
remaining  variables. 

b.  A    "Matched   Set"   Relationship   Exists   Between    the 
Algorithm  and  the  Data  Base 

The  algorithm  for  processing  data  can  only  work  as 

well  as  the  quality  of  the  data  and  vice  versa. 

For  best  matching,  both  components  should  be  at  a 
comparable  level  of  acceptable  perfection. .. .An  upper 
bound  than  on  the  fidelity  of  the  algorithm  would  that  it 
track  the  variables  to  no  greater  degree  of  granularity 
than  that  expected  in  the  data  base.  (Patton,  1990,  pp. 
43-44) 
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c.  The  Study  of  War  Can  Approximate  an  Exact  Science 

Technology  has  advanced  the  art  of  wargaming  and 
reduced  in  degree  many  limitations.  There  is  one  limitation, 
however  that  technology  cannot  impact.  A  person  is  unable  to 
predict  how  they  or  any  other  individual  will  react  in 
stressful  and  dangerous  situations  of  war.  This  inability  to 
predict  human  action  and  reaction  is  a  major  problem  and  a 
fundamental  reason  why  the  social  sciences  are  generally  less 
exact  than  physical  sciences  (Hausrath,  1971,  p.  276). 

d.  The    Doctrine    for    Tactics    and   Data    Used    for    the 
War game  Design   is  Current 

The  rapid  advances  of  technology  which  produces 
new  or  improved  weapons  and  systems  contributes  to  doctrine 
evolution  and  input  data  change.  Continuous  revision  of 
doctrine  and  data  used  in  a  wargame ' s  design  is  not  practical. 
Between  periodic  updates  to  reflect  revised  doctrine  and  data, 
a  wargame  will  be  out  of  date. 

e.  A  Wargame  is  Repeatable 

In  every  human-decision  game,  the  players  learn 
from  experience.  Therefore  with  each  additional  run  of  the 
game,  the  players  have  knowledge  that  they  did  not  have  in  the 
previous  game  and  this  will  affect  their  decisions.  It  is 
still  useful  to  repeat  a  wargame  to  confirm  the  consistency  of 
results  or  the  lack  of  consistency.  In  general  the 
reliability,  validity,  and  applicability  of  the  results  of 
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reliability,  validity,  and  applicability  of  the  results  of 
human-decision  games  are  more  a  matter  of  judgment  than  of 
measurement  (Hausrath,  1971,  p.  287).  In  addition,  if  a 
stochastic  method  of  determining  the  outcome  of  chance  events 
is  used,  the  same  decision  can  result  in  a  different  outcome. 

5.  Cost 

The  four  massive  requirements  of  data,  time,  staff  and 
computer  support  add  up  to  big  costs.  Although  a  wargame  is 
less  expensive  than  a  fleet  exercise,  the  preparation,  conduct 
and  evaluation  time  for  a  wargame  should  not  be 
underestimated.  Pregame  planning  and  postgame  analysis  can 
require  several  months  of  work.  Player  training  and  playing 
time  takes  them  away  from  their  jobs. 

6.  Stochastic  vs.  Deterministic 

A  simplistic  description  of  the  issue  is  whether  it  is 
more  important  to  obtain  expected  results  (firing  200 
things  each  having  a  probability  of  kill  of  0.5  will  cause 
100  kills);  or  whether  it  is  more  important  to  allow  for 
all  possible  outcomes  (firing  200  things  as  above  could 
cause  100  kills,  or  200,  or  zero  etc.).  (Patton,  1990,  p. 
44) 

Some  designers  believe  that  a  game  will  be  more 

realistic  if  chance  is  portrayed  stochastically  so  the  players 

will  be  forced  to  consider  unexpected  and  unusual  outcomes. 

The  opposing  argument  is  that  for  a  game's  very  finite  set  of 

encounters  an  outcome  from  the  tails  of  the  distribution  could 

easily  skew  the  game  results. 
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B.   HAZARDS 

The  following  is  a  list  of  wargaming  considerations  that 
are  hazardous  to  the  accurate  results  of  game  play.  The 
researcher  must  anticipate  and  be  vigilant  to  avoid  these 
conditions  when  selecting  a  game,  when  the  game  is  played  and 
when  representing  conclusions. 

1 .  Poor  Documentation 

The  history  of  modeling  shows  that  poor  documentation 
creates  poor  results.  Modeling  documentation  is  critical  when 
determining  appropriate  applications.  The  ongoing  process  of 
wargame  revision  can  make  the  initial  documentation  obsolete. 
In  addition,  personnel  turnovers  deplete  the  corporate 
knowledge  base  concerning  details  and  assumptions  in  models 
used  (Airpower  Research  Institute,  1985,  p.  35). 

2 .  Uncooperative  Players  and  Umpires 

A  wargame  is  a  competitive  situation.  The  players 
bring  their  egos  to  the  game  and  don't  like  to  lose.  They 
have  pride  and  some  will  improvise. 

In  order  for  the  game  to  be  successful,  the  players 
must  play  wholeheartedly  and  enter  into  the  spirit  of  the 
game.  There  may  be  times  in  the  game  when  a  player  is  not 
busy  and  may  become  bored.  Another  tendency  is  for  some 
players  to  "fight  the  problem"  by  seeing  only  the  limitations 
of  the  game  models  and  not  the  advantages.  A  weapon  may  not 
perform  "correctly"  or  an  outcome  may  not  be  what  they  expected. 
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Umpires  sometimes  have  to  apply  subjective  values 

judgments  in  determining  an  outcome  and  abuses  can  occur. 

Umpires  can  limit  the  information  they  give  to  players  and 
"muddy  the  waters"  with  false  or  inaccurate  data.  Umpires 
can  also  be  arbitrary  and  inconsistent  in  their  handling 
of  such  factors.  Similarly,  there  is  a  strong  tendency 
for  human  umpires  to  lapse  into  reporting  information  in 
game  terms  rather  than  real  terms.   (Perla,  1990,  p.  304) 

3 .   Misplaced  Advocacy 

The  results  of  wargaming  are  used  to  communicate  and 
convince.  The  inherent  ambiguity  of  a  not  rigorously 
quantifiable  problem  allows  misinterpretation  of  wargame 
results.  Because  assumptions  and  data  input  can  be 
manipulated,  there  is  an  opportunity  to  use  the  wargame  to 
produce  a  desired  outcome.  The  designer  of  a  game  has  great 
power  to  inform  or  to  manipulate  (Perla,  1990,  p.  182). 

This  chapter  listed  and  described  the  limitations  that 
cannot  be  eliminated  from  wargaming  to  enable  the  reader  to 
understand  the  restrictions  within  which  a  wargame  attempts  to 
approximate  reality.  It  also  provided  a  list  of  elements  that 
are  dangerous  to  useful  game  results.  The  next  chapter 
contains  a  description  of  the  steps  in  a  general  wargame 
experiment  plan. 
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IV.   GENERAL  EXPERIMENT  PLAN 

This  chapter's  aim  is  to  provide  a  broad  outline  of  and 
guidelines  for  designing  a  wargame  experiment  to  test  a  C3 
hypothesis.  The  reader  should  be  familiar  with  general 
experimental  design  and  analysis  of  variance.  A  one  or  two- 
semester  course  in  statistics  would  be  sufficient. 

Experiment  credibility  hinges  on  the  approach  used  to  plan 
or  design  the  experiment.  A  clearly  defined  approach  helps  to 
ensure  that  the  final  product  is  appropriate.  The  statistical 
design  of  an  experiment  is  the  process  of  planning  the 
experiment  so  that  appropriate  data  will  be  collected.  The 
data  is  then  analyzed  by  statistical  methods  resulting  in 
valid  and  objective  conclusions.  There  are  two  considerations 
to  any  experimental  problem:  the  design  of  the  experiment  and 
the  statistical  analysis  of  the  data.  They  are  related 
because  the  method  of  analysis  depends  directly  on  the  design. 
The  goal  is  a  cost  effective  experiment  that  supports  informed 
decisions. 

The  process  for  planning  an  experiment  includes  the 
following  stages:  conception,  design,  preparation,  execution, 
analysis  and  report.  A  looping  cycle  of  refinement  is  likely 
to  occur  when  going  from  conception  to  design  and  from  design 
to  preparation. 
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The  success  of  an  experiment  is  measured  by  the  degree  to 
which  the  experiment  results  answer  the  requested  issues  and 
objectives,  and  the  credibility  of  those  results.  A  effective 
experiment  requires  a  team  of  militarily  knowledgeable  C3 
personnel  (User)  and  statisticians  that  are  experienced  in 
experimental  design  (Analyst) . 

Although  all  of  the  six  stages  will  be  described,  the 
thesis  is  mainly  concerned  with  the  design  and  preparation 
stages. 

A.   CONCEPTION 

Recognition  of  and  a  statement  of  the  problem  is  the  first 
step.  This  step  includes  going  from  the  issues,  to  the 
objectives  and  then  to  the  response  variables.  The  importance 
of  this  stage  cannot  be  overemphasized.  The  Users  begin  by 
writing  a  detailed  statement  of  the  issues  to  be  investigated 
in  the  experiment.  They  next  define  a  clear,  unambiguous 
description  of  the  specific  objectives  of  the  experiment  and 
how  they  relate  to  the  identified  issues.  Experiment 
alternatives  should  flow  from  and  support  the  objectives.  A 
clear  statement  of  the  assumptions  is  written. 

Relevant  measures  of  effectiveness  (MOE)  and  measures  of 
performance  (MOP)  are  determined  and  designed  to  provide  the 
decision-maker  with  either  a  quantitative  or  qualitative 
assessment  of  the  objectives.  The  environment  and  the  threat 
must  be  chosen  because  they  influence  software  selection. 
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Security  considerations  unique  to  the  experiment  are  also 
determined  during  this  stage.  The  Analyst  can  assist  in 
helping  to  identify  the  experiment  alternatives,  assumptions 
and  write  the  hypothesis.  The  Analyst  makes  sure  that  there 
is  a  clear  linkage  between  the  objective,  alternative,  and 
hypothesis. 

The  Analyst  will  also  assist  the  User  in  selecting  the 
independent  variables.  These  factors  may  be  either 
quantitative  or  qualitative.  How  these  factors  are  to  be 
controlled  and  measured  is  also  considered.  The  levels  of  the 
factors  must  also  be  chosen.  The  dependent /response  variable, 
how  it  will  be  measured,  and  the  probable  accuracy  of  those 
measurements  are  also  determined.  The  User  and  the  Analyst 
must  be  careful  to  be  certain  that  the  response  to  be  measured 
really  provides  information  about  the  objective. 

B.   DESIGN 

The  focus  at  this  step  is  to  match  the  experiment 
objectives  with  available  simulation  models,  analysis  tools, 
data  requirements,  measures,  hardware  requirements,  and 
security.  The  detailed  experiment  plan  should  have  complete 
traceability  back  to  the  originating  issues,  objectives  and 
requirements . 

A  key  aspect  is  determining  whether  the  issues  and 
objectives  lead  to  a  specific  generic  experiment  category. 
Some   generic   experiment   types   include:   demonstration, 
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optimization,  comparison,  evaluation,  sensitivity  and 
prediction.  Deciding  the  experiment  type  can  assist  in 
writing  the  mathematical  model  for  the  experiment  so  that  a 
statistical  analysis  of  the  data  may  be  performed.  Also 
included  in  this  step  is  determining  the  order  in  which  the 
data  will  be  collected  and  the  method  of  randomization.  The 
number  of  replications  to  support  the  analysis  is  decided. 
The  number  of  subjects  (player) ,  from  what  population  they 
will  be  obtained,  and  to  what  condition(s)  each  player  will  be 
exposed  are  planned. 

This  thesis  concentrates  on  the  experiment  design  steps 
that  are  unique  to  wargaming.  Those  steps  are:  designing  the 
scenario,  providing  a  data  base,  and  setting  up  the  software 
program. 

1 .   Scenario 

The  game  begins  by  placing  the  players  in  a  situation 
that  requires  them  to  make  decisions.  The  scenario  or 
situation  sets  the  scene  for  player  decisions  and  provides  for 
specific  updates  in  the  situation  during  the  play.  The 
updates  can  alter  or  influence  the  developing  situation  and 
can  draw  player  responses  to  specific  items  of  interest.  By 
defining  the  setting  and  scope  of  player  decisions,  scenarios 
can  direct  the  course  of  a  game  into  either  very  narrow  or 
fairly  broad  channels,  depending  on  the  game's  goals  (Perla, 
1990,  pg.  204).   The  scenario  can  have  a  major  influence  on 
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player  decisions  so  the  User  needs  to  be  sure  that  the 
scenario  allows  the  players  enough  decision-making  flexibility 
to  let  the  game  meet  the  objectives.  In  order  to  focus  on 
exploring  the  C3  factors  and  reasoning  that  affect  specific 
types  of  decisions,  scenarios  should  allow  players  as  much 
freedom  as  possible  in  making  decisions  by  minimizing 
artificial  restrictions. 

The  User  needs  to  work  closely  with  the  scenario 
designer.  The  designer  must  understand  the  experiment 
objectives  and  how  they  are  to  be  met  in  the  game.  By 
identifying  the  kinds  of  player  activities  and  decision-making 
opportunities  that  are  required  to  meet  the  game's  objectives, 
the  designer  can  ensure  that  those  activities  and 
opportunities  can  happen.  The  scenario  writer  structures  the 
flow  of  game  play  to  allow  the  User's  needs  to  be  met  but 
without  forcing  the  players  to  follow  a  rigid  path.  A 
critical  node  in  the  flow  of  the  game,  at  which  a  player's 
decisions  will  lead  inevitably  down  one  or  another  of  several 
major  alternative  paths  (to  order  a  battle  force  into  a  fjord 
or  to  hold  it  in  the  open  ocean;  to  launch  a  strike 
immediately  or  await  the  enemy's  attack  first),  may  be  called 
a  decision  point  (Perla,  1990,  pp.  208-209).  These  decision 
points  need  to  have  enough  detail  so  that  players  will  have  a 
realistic  range  of  alternatives,  but  the  number  of 
alternatives  needs  to  be  manageable  for  analysis. 
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A  coherent  scenario  must  be  logically  consistent  and 
tie  together  all  the  elements  of  the  experiment  including  the 
originating  issues,  objectives  and  requirements.  Usually,  it 
must  also  be  a  believable  possible  view  of  reality  from  the 
players  perspective.  The  exception  is  when  the  objectives  of 
the  experiment  are  important  enough  that  the  players  are 
willing  to  shelve  disbelief  to  support  exploration. 

The  User  should  review  the  scenario  designer's 
documentation  including  reasons  behind  choice  of  assumptions, 
factors  included  or  excluded  from  the  scenario,  the  use  of 
particular  sources  of  information,  and  any  other  decision. 

Control  personnel,  players  and  analysts  each  have 

their   own  perspective   and  requirements   for   information 

concerning  the  scenario. 

Control  personnel  need  to  understand  the  context  of  the 
game  and  the  prerogatives  and  limitations  under  which  they 
must  operate.  Players  need  the  same  information  as 
control  personnel,  but  it  should  be  constrained  to  reflect 
their  less-than-perfect  knowledge  of  their  game  world. 
Analysts  need  to  know  not  only  the  full  story  of  what  and 
why,  the  "ground  truth,"  but  also  the  story  as  told  to 
players  and  control,  so  that  they  may  interpret  the 
effects  of  information  constraints.  Finally,  the  future 
consumers  of  the  game's  issues  must  know  not  only  the 
context  of  the  game,  but  also  how  to  distinguish  scenario 
input  from  game-play  output.   (Perla,  1990,  p.  211) 

The  User  also  needs  to  plan  how  to  distribute  the 
scenario  information  to  the  game-control  personnel,  game 
players,   game  analysts  and  report  readers. 
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2.   Data  Base 

In  addition  to  understanding  the  situation,  the 
players  must  have  access  to  the  game's  data  base.  The  data 
base  includes  all  the  information  that  the  players  would 
reasonably  have  access  to  in  a  real  situation.  The  data  base 
of  the  game  contains  information  about  the  capabilities  and 
number  of  forces,  levels  of  logistics,  and  relative 
likelihoods  of  the  occurrence  and  outcome  of  interactions 
between  forces,  and  the  physical  or  environmental  conditions. 
It  also  is  the  doctrine,  tactics  and  procedures  for  the  use  of 
the  forces  and  equipments. 

The  data  base  links  the  scenario  and  the  mechanics  of 
the  game.  It  must  provide  all  the  inputs  required  to  allow 
the  game's  models  to  reproduce  the  qualitative  scenario 
conditions  and  to  generate  outcomes  of  interactions. 

The  data  base  provided  to  the  players  should  be 
tailored  to  their  game  role,  to  the  types  of  decisions  they 
should  be  making,  and  to  the  types  of  information  they  need  to 
make  those  decisions  (Perla,  1990,  p.  212).  Important  data 
need  to  be  easy  to  find  or  require  a  minimum  of  calculations. 
The  players  need  enough  information  to  be  able  to  determine 
what  possible  outcomes  will  result  from  the  various  decisions 
they  may  make. 

Some  data  is  easily  quantifiable  and  available  such  as 
logistics  input  data,  weapon  and  equipment  performance  that 
have   been   experimentally   field   tested,   and   information 
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available  from  records  such  as  ship  transit  time.  Some  data 
will  be  unavailable  such  as  for  weapons  never  used  in  combat 
(tactical  nuclear  weapons).  This  data  must  be  deducted  or 
extrapolated  from  related  data  or  assumed  to  be  within 
reasonable  limits. 

Qualitative  data  for  judgments  of  unit  discipline  and 
morale,  leadership  of  opposing  commanders  can  be  either 
assumed  equal  for  opposing  teams  or  can  be  assigned  quantified 
values  that  represent  different  levels  or  degrees  of  these 
qualities. 

The  C3  data  base  representation  of  doctrine,  tactics 
and  procedures  is  difficult  to  include.  They  change  over  time 
and  sometimes  are  not  well  documented  because  they  depend  on 
the  commander  and  his  own  command  structure.  The  command  and 
control  data  base  should  include  the  communication,  EMCON, 
surveillance,  EW  and  OPCON  matrix. 
3.   Software  (Wargame) 

When  selecting  the  wargame,  the  User  and  the  Analyst 
must  constantly  evaluate  how  the  experiment  objectives 
determined  during  the  conception  stage  will  be  applied  in  the 
game.  This  section  lists  and  describes  areas  that  the  User 
and  Analyst  will  examine  in  designing  the  game.  These 
decisions  form  the  game  parameters.  The  software  to  use  is 
the  one  that  comes  closest  to  these  parameters. 
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a .  Obj ectives 

Every  wargame  has  objectives.  The  objectives 
determine  the  game's  uses.  For  example,  a  game  may  be 
conducted  to  provide  decision-making  experience  at  one  or  more 
specified  levels  and  types  of  command;  another  to  provide 
information  and  data  concerning  the  employment  of  specific 
forces  or  weapons  systems,  test  an  organization  or 
distribution  system,  or  evaluate  a  type  of  operation  or  a 
tactical  doctrine  (McHugh,  1966,  p.  1-11).  The  game  designer 
should  have  clearly  identified  how  and  in  what  ways  the  game 
could  provide  the  type  of  experience  needed  to  achieve  its 
objectives.  The  User  needs  to  match  as  closely  as  possible 
the  experiment  objectives  with  the  game  objectives. 

b.  Scope  and  Level 

The  selection  of  the  scope  and  level  for  an 

experiment  follow  directly  from  the  specific  experiment 

objectives. 

War  games  may  range  all  the  way  from  a  contest  between  two 
units  of  a  single  service  to  a  simulated  global  conflict 
involving  coalitions  of  nations,  the  efforts  of  all 
services,  and  the  impact  of  conventional  and  nuclear 
weapons  on  military  forces  and  civilian  economies.  Games 
may  be  tactical,  strategical,  or  a  mixture  of  both.  Some 
emphasize  air  operations;  others,  land  or  sea  operations. 
Geographically,  games  may  embrace  a  limited  area,  a  single 
area  of  operations,  or  several  areas  of  operations. 
(McHugh,  1966,  p.  1-13) 

Related  to  the  scope  of  a  game  is  the  range  of  command  levels. 
Command  levels  can  range  from  individual  unit  commanders  to 
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national  leaders.  The  scope  and  range  of  command  levels  help 
to  determine  the  basic  military  units  that  will  be  used  in  the 
game.  If  the  lowest  command  level  of  naval  forces  is  the 
squadron,  then  the  smallest  unit  might  be  the  division. 

c.  Number  of  Sides 

Games  can  be  one-,  two-,  and  n-  or  multi-sided. 
A  one-sided  game  is  played  under  two  or  more  different 
situations.  Some  examples  are:  to  find  the  minimum  time  for 
the  simulation  of  a  replenishment  operation  when  there  is  no 
opposition  from  the  enemy;  and  when  the  players  are  opposed  by 
the  game  controllers  or  a  pre-programmed  opponent.  The  most 
common  wargame  is  the  two-sided  with  teams  of  players 
representing  each  of  the  two  major  contending  parties  or 
coalitions  (Blue  vs  Red) .  A  multi-sided  game  is  less  common 
and  is  most  often  used  for  diplomatic  or  political-military 
games. 

d.  Amount   of  Intelligence 

An  "open"  game  is  when  all  players  have  access  to 
complete  information  on  each  other's  plans  and  forces. 
Advantages  to  the  open  game  are  a  reduction  in  personnel, 
space  and  time  requirements.  A  "closed"  game  is  when  the 
players  have  access  only  to  intelligence  that  would  be 
available  in  real-world  conditions.  The  data  generated  in  a 
closed  game  will  be  more  reliable  if  the  simulated  flow 
information  is  incomplete  or  interrupted. 
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A   closed   game   with   information   handling 

enhancements  will  make  C3  experiments  more  credible.  The  best 

approach  for  a  wargame  to  take  in  information  uncertainty  is 

to  have  more  than  one  level  of  truth.   Some  information  is 

progressively  disclosed  automatically  as  the  game  progresses, 

but  more  detailed  and  accurate  information  can  be  acquired  by 

aggressive  sensor  use,  additional  information  requests,  and 

revealed  by  team  members  sharing  their  information.   The  fog 

of  war  can  be  simulated  by  controlling  the  treatment  of 

information. 

Ideally  the  game  should  allow  each  player  to  make  the 
following  "mistakes":  (1)  Non-observation  of  entitles 
which  are  within  detection  range  because  of  non-alerted 
operators,  system  failures,  environment,  and  deception  (or 
loss  of  track  after  initial  detection);  (2)  Incorrect 
location  of  detected  entities  due  to  system  bias,  time 
late,  lack  of  gridlock,  etc.  (multiple  tracks);  (3) 
Incorrect  classification  of  detected  entities  due  to 
noise,  jamming,  environment,  deception,  failure  to 
correlate  separate  sources,  delay,  etc.  (confusion  of  red, 
blue  and  neutral);  (4)  Failure  to  report  events.... In 
addition  to  the  errors  of  the  above  type,  the  gaming 
system  should  introduce  mistakes  of  the  "false  target" 
type.  These  include  radar,  sonar  and  visual  and  ECM 
sightings.   (Sovereign,  1983,  pp.  12-13) 

All  of  the  above  can  lead  the  decision  maker  to  an  incorrect 
assessment  of  the  tactical  situation.  By  allowing  the 
mistakes  to  happen,  C3  is  more  realistically  represented. 
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e.  Methods  of  Evaluation 

One  method  is  called  "free"  umpiring  and  is 
dependent  on  the  experience,  the  judgment,  and  the  objectivity 
of  the  game  controllers.  An  umpire  can  select  from  among  a 
broad  range  of  realistic  outcomes  the  one  that  is  not  only 
probable,  but  that  also  furthers  the  purpose  of  the  game 
(McHugh,  1966,  p.  1-18).  The  advantage  is  a  quick  and  simple 
method  of  evaluation. 

Rigid  umpiring  rules  are  based  on  data  that 
reflect  real-world  interactions.  Contacts,  number  of  hits, 
and  the  amount  of  damage  are  determined  by  equations  and 
computations  performed  manually  or  by  computer. 

Semirigid  evaluation  combines  both  free  and  rigid 
techniques.  Some  situations  are  evaluated  by  qualified 
officers  and  others  are  determined  by  equations. 

f.  How  Time   is  Represented 

Time  is  a  critical  aspect  of  any  wargame  for  two 

reasons. 

First,  in  reality,  timing  and  speed  of  execution  are  often 
decisive  in  determining  the  success  or  failure  of  a 
military  operation.  Second,  time  management  in  a  game 
very  often  determines  the  extent  of  activity  that  the  game 
can  explore.   (Perla,  1966,  p.  222) 

When  a  game  progresses  by  predetermined  and 
constant  intervals,  it  is  a  time-step  game.  Force  movement  is 
conducted  during  the  time  interval  and  at  the  end  of  the 
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interval,  interactions  are  evaluated  and  the  status  of  forces 

is  updated.   For  example,  a  game  may  define  a  single  move  to 

represent  an  entire  day's  operations.   The  shorter  the  time 

interval,  the  longer  the  time  required  for  play. 

Another  approach  to  time-increment  moves  is  to 

vary  the  amounts  of  real  time  depending  on  the  importance  and 

intensity  of  activity  expected  during  a  given  time  span. 

For  example,  a  pre-hostilities  move  may  encompass  ten  to 
fifteen  days  of  activity.  The  D-day  move  of  the  same  game 
may  represent  just  a  single  day.   (Perla,  1990,  p.  224) 

The  time  interval  for  various  moves  may  be  predetermined  in 
game  design  or  may  be  defined  by  the  game  controllers  during 
the  game. 

A  critical  event  or  event  store  game  advances  play 
from  event  to  event. 

Another  method  is  continuous  time.  Player 
decisions  may  be  made  at  any  time  and  the  orders  and  force 
attempts  to  execute  those  orders  are  continuous.  Continuous 
time  is  closer  to  reality  and  is  more  likely  to  encourage 
dynamic  interactions  that  occur  in  real  operations.  The  game 
may  be  played  faster,  at  the  same  rate,  or  slower  than  real 
time.  However,  when  the  game-time  to  real-time  ratio  is  not 
one-to-one,  there  is  a  potential  for  distortion  in  the 
planning  process.  When  the  game  is  speeded  up,  planning  time 
is  reduced  (uses  too  much  game  time)  and  when  the  game  is 
slowed,  decision  time  pressure  is  unrealistically  reduced. 
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g.  Method  for  Treating  Chance  Events 

Since  chance  events  occur  in  real  war,  a  wargame 
must  be  designed  to  deal  with  the  probability  that  particular 
conditions  will  occur  and  to  the  variety  of  possible  effects 
that  will  result  from  a  given  set  of  actions.  Two  ships  may 
be  adversaries.  Which  ship  will  fire  first?  Which  ship  will 
get  the  first  hit?  Will  the  first  hit  inflict  little  damage, 
cripple  or  sink  the  ship?  What  will  happen  to  the  opposing 
ship  in  the  meantime?  This  example  of  a  simple  action  has  a 
whole  set  of  probabilities  or  chance  outcomes.  For  many 
events,  the  probabilities  have  been  determined  from  war, 
exercises  and  test  data.  When  the  data  is  not  available, 
best-estimate  values  are  made  by  qualified  experts.  Wargames 
must  attempt  to  duplicate  this  complex  of  elements  and  factors 
that  may  occur  in  almost  infinite  variety  in  real  war 
(Hausrath,  1971,  p.  285). 

There  are  two  types  of  game  uncertainties: 
stochastic  and  deterministic.  Stochastic  uncertainties  are 
often  characterized  as  the  "roll  of  the  dice",  have  a 
probability  distribution  and  use  random  numbers. 
Deterministic  uncertainties  have  a  known  set  of  inputs  which 
result  in  a  unique  set  of  outputs,  and  the  probabilities  of 
the  occurrence  is  an  average  or  expected  value.  Each  model 
will  usually  require  a  combination  of  the  two  types  of 
uncertainties  and  their  correct  application  is  critical. 
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Because  many  assessments  are  based  on  chance 

factors,  the  Analyst  should  carefully  evaluate  the  reliability 

of  results  by  first  performing  a  sensitivity  analysis. 

Wide  deviations  may  occur  within  the  data  from  which 
average  or  best-estimate  values  are  derived.  The  ranges 
of  variation  of  these  values,  within  which  the  results 
will  remain  valid,  i.e.,  within  which  the  values  can  vary 
without  changing  the  nature  of  the  result,  are  determined 
by  sensitivity  analysis. .. .Sensitivity  analysis  does  not 
correct  the  data  or  assumptions  used;  it  does  give  the 
analyst  some  measure  of  the  level  of  confidence  that  can 
or  should  be  associated  with  derived  findings  or  results. 
(Hausrath,  1971,  p.  284) 


h.      Models 

As  defined  in  Chapter  I,   a  model  can  be  a 

description  of  a  system,  a  process,  or  a  series  of  related 

events.     The  macro  model   of   a  wargame   is   a  complex 

configuration   of   micro   models   and   an   explanation   or 

description  of  their  interrelationships.    The  most  common 

micro  models  for  wargames  are  weapons,  logistics,  command  and 

control,  intelligence,  communications,  physical  environment, 

and  sensors.   During  game  play,  the  models  can  be  programmed 

to  calculate  decision  results. 

If  the  scenario  sets  the  stage  for  the  game,  and  the  data 
base  provides  the  information  required  for  players  to  make 
decisions,  the  game's  mechanics  allow  those  decisions  to 
be  implemented  and  inform  the  players  about  their  effects. 
Wargame  mechanics  may  be  considered  as  two  interrelated 
systems:  models  and  procedures.  The  models  translate  data 
and  decisions  into  game  events.  Procedures  define  in  game 
terms  what  players  can  and  cannot  do  and  why,  sequence  the 
game  events  to  allow  for  accurate  recreation  of  cause  and 
effect,  and  manage  the  flow  of  information  to  and  from 
players  and  control.   (Perla,  1990,  p.  214) 
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All  models  should  be  documented  so  the  User  can  understand  the 
assumptions  and  algorithms  contained  within  the  model  design. 

A  definition  of  the  C3  model's  function  is  to 
plan,  direct,  coordinate,  and  control  assigned  and  attached 
forces  to  accomplish  a  mission.  Many  C3  models  accomplish 
this  function  by  using  a  combination  of  humans  controlling  the 
higher  echelon  decisions  and  computer  generated  decisions  at 
lower  levels.  Command  and  control  processes  require  the  use 
of  the  communication  model  and  often  act  on  information  from 
the  intelligence  and  sensor  models.  Command  and  control 
processes  normally  include  orders  to  subordinates,  queries  to 
superiors  and  subordinates  for  information,  and  an  information 
flow  to  update  superiors  and  subordinates. 

Basic  C3  models  include  the  following  entities 
with  their  attributes: 


•  C2  units:  location;  type  of  combat  unit  representative  of 
this  headquarters  unit;  lower  echelon  units  under  the 
direction  of  this  unit;  description  of  the  echelon  that 
the  headquarters  represents;  its  nationality  and  its 
vulnerability  to  various  kinds  of  attack 

•  personnel:  authorized  quantity;  on-hand  quantity;  fraction 
of  the  on-hand  quantity  available  for  duty;  a  factor 
representing  a  fatigue  coefficient;  and  a  capability 
factor 

•  communications:  its  own  organic  communications  facilities; 
access  to  communications  facilities;  or  have  a  combination 
of  both 

•  facilities:  size,  vulnerability,  mobility,  level  of 
capability  (to  determine  response  time),  and  capacity 
(National  Defense  University,  1987,  p.  II-4). 
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These  entities  and  their  functions  require  the  support  of  all 

the  other  models  and  need  a  direct  or  indirect  interface. 

i.  Validation   and  Verification  Considerations 

Validation  is  a  measure  of  how  much  confidence 

there  is  on  the  output  of  a  given  model.   There  are  two 

approaches  for  validation.   The  first  approach  evaluates  the 

input  data,   the  basic  principles  of  the  model  and  its 

assumptions.     The   second   method   collects   real   world 

information  and  compares  it  with  the  output  of  the  wargame. 

One  of  the  most  important  duties  of  the  User  and  Analyst  team 

is  to  determine  the  validity  of  the  game's  results  and 

processes. 

The  validity  of  a  game's  results  can  best  be  thought  of  as 
the  extent  to  which  those  results  reflect  reality  as 
opposed  to  the  artificiality  of  the  gaming  environment. 
This  validity  depends  on  the  accuracy  of  the  mathematical 
evaluations  of  operational  capabilities  used  in  the  models 
of  the  game,  and  on  how  well  the  quantity  and  quality  of 
information  available  to  the  players  of  the  game  reflect 
the  levels  of  information  likely  to  be  available  to  their 
real-life  counterparts.   (Perla,  1990,  p.  236) 

Assumptions  and  input  data  need  to  be  carefully 
examined.  The  results  can  be  no  better  than  the  validity  of 
the  data  and  the  assumptions  from  which  they  are  derived. 

The  User  also  needs  to  understand  the  quality  or 
utility  of  both  the  macro  and  micro  models.  This  is 
verification:  is  the  model  working  as  advertised.  In  the 
course  of  game  play,  are  the  experiences  and  insights  the 
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players  may  gain  from  it  determined  primarily  by  the  game's 
scenarios,  assumptions,  and  mathematical  combat  models,  or  by 
the  decisions  that  the  players  are  capable  of  making.  When 
the  game  is  dominated  by  its  mechanics  instead  of  the 
decisions  choices,  misconceptions  and  errors  in  understanding 
can  result.  Player  decisions  cannot  be  driven  from  scenario 
mandates  and  constraints.  An  example  of  a  player  decision 
choice  constrained  by  the  scenario  is  assigning  a  mission  to 
deter  war  in  a  crisis  but  mandating  all  the  answers  to  the 
strategic  questions.  The  key  to  creating  accurate  models  and 
systems,  then,  lies  in  the  ability  of  the  design  to 
incorporate  as  many  of  the  elements  important  to  actual 
decision  makers  as  possible  while  simultaneously  minimizing 
the  number  and  effect  of  extraneous  factors  (Perla,  1990,  p. 
242)  . 

In  addition  to  reviewing  the  model  developer's 
test  plan  and  results,  the  users  manual,  the  algorithm 
document  including  where  they  came  from  or  how  they  were 
derived,  the  User  can  run  tests  and  verify  that  the  overall 
game  model  will  work  for  their  experiment. 

4.   Support  System  Capability  Requirements 

The  capabilities  of  a  wargaming  support  system  should 
be  considered.  The  system  should  have  a  wide  range  of 
capabilities  including: 

•  Multiple  simultaneous  users 
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•  Multiple  interactive  consoles 

•  File  management  for  large  data  bases 

•  Security  for  file  access. 

•  Graphic  displays 

•  Recording  and  replay 

This  list  should  be  considered  as  a  minimum  and  additions  will 
be  dictated  by  the  experiment  plan. 

C.   PREPARATION 

1.  Schedule 

A  final  schedule  of  experiment  activities  should  not 
be  established  until  all  aspects  of  conception  and  design  have 
been  determined.  Time  must  be  planned  for  writing  the 
scenario,  gathering  and  in-putting  the  data,  player  training 
pretesting,  running  the  experiment  and  reserving  facilities 
and  hardware. 

2.  Players  and  Control  Personnel 

The  number  of  players  on  a  team  and  their  decision 
making  levels  need  to  be  adequate  to  meet  the  experiment/game 
objectives.  The  player  roles  need  to  be  consistent  with  the 
geographic  and  operational  scope  and  scale  of  the  game.  Each 
player  needs  to  have  a  well-defined  operational  role  and  must 
understand  their  function  within  the  group.  Players  in  the 
roles  of  non-U. S.  or  threat  roles  may  need  special  training  or 
expertise.    They  must  not  only  understand  the  technical 
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capabilities  of  their  foreign  weapons  but  also  the  tactical 
and  strategic  doctrine  of  the  opposition. 

Command  levels  above  and  below  the  team  level  may  be 
assumed  by  the  game  controllers.  At  this  stage,  the  players 
and  control  personnel  are  identified  and  matched  to  the 
functions  to  be  performed  to  conduct  the  game. 

Player  and  control  personnel  training  and 
indoctrination  are  performed  at  this  time. 

3 .  Forms  and  Game  Directives 

The  special  data-recording  forms  and  work  sheets  must 
be  designed  for  the  experiment  and  duplicated.  The  game 
directives,  scenario,  order-of -battle  information  and  other 
materials  used  as  preliminary  information  handouts  to  players 
need  to  be  prepared  and  duplicated. 

4.  Playtest 

Playtesting  is  the  best  way  to  validate  the  entire 

game  system  by  verifying  that  all  the  micro  models  mesh 

together  to  form  a  workable  macro  model.  As  the  word  implies, 

playtesting  combines  playing  the  game  by  thoroughly  testing 

its  functions  and  its  ability  to  meet  its  objectives  (Perla, 

1990,  p.  242).  To  ensure  that  the  game  design  is  complete  and 

that  all  data  and  procedures  needed  are  available  and  correct, 

the  Analyst  thoroughly  tries  out  the  various  alternatives  and 

attempts  to  make  the  game  fail. 

Especially  important  is  the  system's  response  to  unusual, 
unexpected,  or  extreme  decisions  players  may  make.   This 
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is  especially  important  for  system  games,  particularly 
those  controlled  by  computer,  in  which  the  stabilizing 
influence  of  a  human  umpire  is  missing.  The  player  that 
'cracks  the  code'  of  the  game  may  play  the  game  very 
successfully,  but  the  educational  value  of  his  experience 
and  the  operational  insights  to  be  obtained  from  his 
decision  making  may  be  nonexistent  because  he  is  only 
taking  advantage  of  a  loophole  that  exists  in  the  game 
system  but  would  not  exist  in  reality.  (Perla,  1990,  p. 
234) 


This  testing  process  enables  the  User  to  correct  problems  with 
the  system,  procedures,  or  data  before  the  experiment  begins. 
Preplaying  also  gives  the  game  controllers  an  opportunity  to 
familiarize  themselves  with  the  game  mechanics  and  their 
responsibilities . 

D.   EXECUTION 

This  is  the  actual  game  playing  and  data  collection 
process.  In  a  two-sided  game,  each  team  makes  the  decisions 
and  perform  actions  appropriate  to  their  mission.  The 
movements,  actions,  and  interactions  are  recorded  by  the 
computer.  The  game  controllers  have  three  functions.  First 
is  to  monitor  the  actions  of  the  teams  and  to  enforce  the 
rules  of  the  game.  Second  is  to  evaluate  the  interactions 
following  the  methods  and  data  of  the  game  and  the  third 
function  is  to  answer  queries  from  the  teams  with  the 
information  and  intelligence  that  are  appropriate.  The  cycle 
for  each  team  and  controller  interaction  is:  team  decision  - 
team  action  -  controller  monitor  -  controller  evaluate  - 


48 


controller  inform.  The  game  controllers  need  apply  the  rules 
in  as  uniform  an  experimental  environment  as  possible  for  each 
game  run . 

It  is  very  useful  to  conclude  the  experiment  by  requesting 
an  overall  assessment  of  the  game  and  for  specific  comments 
about  particular  issues  or  aspects  of  play  from  the  team 
members . 

E.  ANALYSIS 

Experiment  data  was  generated  in  various  formats, 
quantities  and  data  rates.  In  accordance  with  the  objectives 
of  the  game,  the  data  generated  in  the  play  of  the  game  will 
be  sorted,  compiled,  and  the  results  interpreted.  A  number  of 
different  analytical  techniques  may  be  used  and  were  selected 
in  the  conception  stage.  In  this  step,  the  Analysts  apply  the 
chosen  methods  of  data  analysis  to  the  experiment  data. 

F .  REPORT 

There  are  three  common  report  formats  of  which  the  User 
may  request  any  or  all.  The  first  report  is  a  quick  look.  It 
is  written  as  soon  as  possible  after  the  experiment  and 
summarizes  the  actual  conduct  of  the  experiment  as  compared  to 
the  planned  procedures.  This  report  also  advises  as  to 
whether  the  information  gained  will  be  applicable  to  the 
issues  and  objectives  of  the  experiment  and  any  possible 
problems  anticipated  in  performing  the  data  analysis. 
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A  preliminary  report  can  be  provided  no  later  than  two 
weeks  after  experiment  completion.  It  supplies  more  detail 
concerning  the  information  provided  in  the  quick  look  report. 
Benefits  of  the  preliminary  report  include  a  preview  of  the 
format  and  completeness  of  the  final  report  so  the  User  can 
provide  an  opinion. 

The  final  report  includes  an  assessment  of  whether  or  not 
the  objectives  of  the  experiment  were  accomplished  and  the 
conclusions.  It  can  include  an  appraisal  of  the  credibility 
of  the  experiment,  recommendations  for  follow-on  analyses  of 
the  data  and  follow-on  experiments  to  more  fully  examine  the 
experiment  and/or  spin-off  issues,  and  lessons  learned 
concerning  the  experimental  processes,  resources  used,  and 
design  and  analysis  techniques. 

This  chapter  described  the  stages  of  the  process  for 
planning  an  experiment  using  wargaming.  It  concentrated  on 
the  experiment  design  steps  that  are  unique  to  wargaming: 
designing  the  scenario,  providing  the  data  base,  and  setting 
up  the  software  program.  It  is  followed  by  an  example  of  a  C3 
experiment  which  was  planned  to  evaluate  a  decision  aid.  A 
description  of  this  experiment  provides  an  opportunity  for  the 
reader  to  understand  the  application  of  the  principles  given 
in  this  chapter. 
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V.   EXAMPLE  EXPERIMENT 

This  chapter's  intent  is  first  to  provide  the  reader  with 
an  example  of  an  actual  C3  experiment  by  summarizing  the 
report  into  the  following  stages:  conception,  design, 
preparation/execution,  analysis  and  report.  Then  a  list  of 
evaluation  questions  and  answers  is  given  for  appraising  the 
example  experiment  plan.  Although  very  little  information 
concerning  the  wargame  is  provided  in  the  report,  a  list  of 
questions  for  a  wargame  critique  is  also  provided. 

This  C3  experiment  investigates  the  effects  of  introducing 
a  decision  aid  into  a  headquarters,  and  was  conducted  for  the 
Defense  Communications  Agency  (DCA)  and  the  Naval  Ocean 
Systems  Center  (NOSC)  by  ALPHATECH,  Inc.  The  objective  of  the 
experiment  was  to  learn  how  the  Naval-Based  Replanning  System 
(N-KRS),  a  decision  aid  for  planning  Naval  air  strikes  from 
aircraft  carriers  against  land  targets,  can  best  support  the 
strike  planning  process.  The  experiment  used  the  tactical 
naval  warfare  simulation  designated  Research,  Evaluation,  and 
System  Analysis  (RESA)  and  which  is  located  at  the  wargaming 
facility  at  NOSC. 

The  Headquarters  Effectiveness  Assessment  Tool  (HEAT)  was 
applied  in  this  experiment  to  validate  defensible  conclusions. 
The  HEAT  measurement  tool  consists  of  a  set  of  variables 
(effectiveness  measures)  and  a  rigorous  set  of  methods  and 
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procedures  for  applying  the  measures  to  experiments  and  for 
analyzing  the  results.  HEAT  measures  fall  into  three  broad 
categories;  process,  performance/  and  effectiveness.  The 
measures  are  based  on  a  theory  of  headquarters  decision  making 
operating  in  the  following  cycle:  gathering  information, 
assessing  the  situation,  generating  decision  options, 
predicting  the  outcomes  of  each  option,  selecting  a  response, 
executing  the  response,  monitoring  its  outcome  and  then 
repeating.  The  HEAT  methodology  was  originally  developed  for 
DCA  eight  years  ago  to  understand  the  headquarters  planning 
process.  The  experiment  described  here  extended  the  use  of 
HEAT  to  examine  the  decision  making  issues  associated  with  the 
introduction  of  a  decision  aid  to  support  planning  in  a 
headquarters  environment  (ALPHATECH,  Inc.,  1989,  pp.  1-2). 
This  chapter  is  a  summary  of  the  experimental  report  prepared 
by  ALPHATECH,  Inc.  The  information  presented  is  typical  of 
the  data  available  after  an  experiment. 

A.   CONCEPTION 

Recognition  of  and  a  statement  of  the  problem  is  the  first 
step.  A  detailed  statement  of  the  issues  to  be  investigated 
in  the  experiment  is  written. 

1.   Navy  Air  Strike  Planning  Problem 

The  purpose  of  Navy  air  strike  planning  is  to  achieve 
a  mission  goal  while  satisfying  a  number  of  resource 
constraints.   Tradeoffs  exist  between  weapon  effectiveness, 
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weapon    availability,    aircraft    capability,    aircraft 

availability,  fuel  requirements,  and  other  factors.   The  air 

strike  planner  may  receive  guidance  from  higher  authority  on 

the  target  priorities.   In  the  experiment,  the  players  were 

given  a  desired  probability  of  a  kill  (Pk)  for  the  targets  to 

indicate  the  priority  of  the  target,  with  a  higher  Pk 

indicating  a  higher  priority.  This  is  an  example  of  an  aspect 

of  the  experiment  design  which  did  not  follow  reality. 

A  typical  planning  process  (as  observed  in  the  experiment) 
starts  by  considering  the  nature  of  the  target  and  the 
desired  Pk,  and  calculating  the  bombs  required  to  achieve 
that  Pk  ( weaponeer ing) .  The  aircraft  needed/available  to 
transport  those  bombs  is  then  considered,  and  fuel 
requirements  for  the  bombing  mission  are  calculated.  The 
aircraft  needed  for  SAM  suppression,  fighter  escort,  and 
jamming  missions  are  considered,  and  total  fuel 
requirements  are  calculated.  If  the  total  fuel  needed 
exceeds  the  fuel  available,  then  the  planning  process 
recycles,  with  changes  being  made  until  the  plan  is 
satisfactory.  Planning  is  not  necessarily  done  in  this 
sequence,  however.  The  planning  process  can  start  with 
any  aspect  of  the  strike  as  long  as  all  of  the  relevant 
factors  are  eventually  considered.  (ALPHATECH,  Inc.,  1989, 
pp.  4-5). 

A  number  of  factors  in  the  air  strike  planning  process 
were  not  included  in  the  experiment.   Examples  are  weather, 
navigation  overflight  restrictions  and  launch  sequencing. 
2.   Navy  Knowledge-based  Replanning  System  (N-KRS) 

The  N-KRS  decision  aid  was  produced  by  NOSC 
specifically  for  Naval  air  strikes  and  was  based  on  the  KRS 
strike  planning  decision  aid  developed  by  the  Air  Force.   The 
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decision  aid  checks  for  consistency  in  air  strike  plans  and 

tracks  resource  availability  and  resource  use. 

N-KRS  allows  the  planner  to  specify  the  goals  of  an  air 
strike  and  then  suggests  a  resource  allocation  to  meet 
those  goals  within  the  constraints  of  the  resources 
available.  The  user  interface  is  in  the  form  of  a 
"template"  with  9  primary  "slots."  The  values  assigned  to 
these  slots  constitute  the  strike  plan.  The  user  may 
specify  values  for  any  number  of  the  9  slots.  N-KRS  can 
function  in  a  fully  automated  mode,  in  which  the  user 
specifies  only  the  target  for  the  strike  and  N-KRS  fills 
in  all  of  the  remaining  slot  values  based  on  available 
resources  and  a  knowledge  base  of  planning  rules.  It  can 
also  function  in  a  fully  interactive  mode,  in  which  the 
user  specifies  values  for  all  9  slots,  and  the  decision 
aid  merely  verifies  that  the  plan  is  consistent  and  does 
not  violate  resource  constraints.  (ALPHATECH,  Inc.,  1989, 
p.  9) 


N-KRS  recognizes  a  number  of  different  mission  types. 
The  relevant  types  for  the  experiment  were  Power  Projection 
Air  (PPA)  missions  (bombing  missions),  SAM  Suppression 
missions,  Air  Escort  missions,  electronic  countermeasures 
(ECM) ,  and  refueling.  The  following  slots  in  the  N-KRS  user 
interface  were  used:  target,  Pk,  ordnance,  aircraft,  AC 
number,  carrier,  unit  (squadron),  time  of  launch  and  time  over 
target. 

3.   Experiment  Objectives 

The  experiment  had  three  primary  objectives: 

1.   Evaluate  the  value  of  the  N-KRS  decision  aid  for  Naval 
air  strike  planning.   Specific  relevant  questions  are: 

(a)  Does  the  N-KRS  have  a  measurable  effect  on  air  strike 
outcomes? ; 
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(b)  How  does  the  N-KRS  assist  the  air  strike  planner?  - 
reduce  workload,  -  reduce  time,  -  reduce  errors,  -  allow 
more  option  consideration,  -  allow  more  recent  information 
incorporation . 

2.  Demonstrate  the  N-KRS  decision  aid  to  experienced  air 
strike  planners  and  obtain  feedback  on  potential 
applications  and  areas  for  improvement. 

3.  Apply  and  improve  the  HEAT  methodology  for  collecting 
observational  data  on  a  commander's  decision  making 
processes  and  linking  process  measures  to  outcome  measures. 
Add  to  database  of  results  within  the  HEAT  framework. 
(ALPHATECH,  Inc.,  1989,  p.  2) 

The  objectives  need  to  directly  relate  to  the  issues 
of  the  experiment. 

B.   DESIGN 

1 .  Overview 

The  objective  of  the  experiment  was  to  compare  air 
strike  planning  with  and  without  the  N-KRS  to  learn  how  the 
decision  aid  can  best  support  air  strike  planning.  In  the 
experiment,  four  players  prepared  air  strike  plans  with  and 
without  N-KRS,  and  their  strike  plans  were  simulated  using  the 
RESA  wargame.  Data  on  aircraft  lost  during  the  strike  and 
damage  to  the  targets  were  generated  by  RESA  and  provided  to 
the  subjects  for  use  in  planning  subsequent  strikes 
(ALPHATECH,  Inc.,  1989,  p.  12). 

2 .  Experiment  Conditions 

There  were  two  major  factors  in  the  experiment:  (1) 
the  presence  or  absence  of  the  N-KRS  decision  aid.  Each 
factor  had  two  levels:  the  difficulty  of  the  scenario  -  easy 
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or  hard.  The  design  was  a  two-by-two  factorial  with  four 
conditions;  each  player  participated  in  all  four  conditions. 
In  order  to  separate  learning  effects  over  time  from  the 
effect  of  using  the  decision  aid,  two  of  the  players  began  the 
experiment  in  the  manual  condition  and  two  began  by  using  the 
decision  aid  (ALPHATECH,  Inc.,  1989,  p.  12). 

For  the  easy  scenario,  initial  planning  was  based  on 
the  player's  assessment  of  the  resources  needed  for  the 
mission.  After  the  simulation  was  completed,  the  player 
planned  a  second  wave  and  had  to  take  into  account  battle 
damage  assessment  and  any  losses  of  aircraft  due  to  an  enemy 
counterattack  before  the  second  launch.  The  difficult 
scenario  was  the  same;  except  in  planning  the  second  attack, 
the  player  had  to  consider  that  the  enemy  had  moved  a  number 
of  its  aircraft  away  from  the  target  base.  Also,  the  enemy 
counterattack  on  the  carrier  group  used  more  resources  than  in 
the  easy  scenario.  Additionally,  the  players  planned  a  third 
strike  that  required  them  to  assess  the  aircraft  available 
from  the  first  strike  (taking  losses  into  account)  and  the 
battle  damage  from  the  second  strike.  The  third  strike  was 
not  played  out  in  RESA. 

Players  in  the  experiment  planned  an  air  strike 
against  three  enemy  air  bases,  using  the  resources  of  three 
Navy  aircraft  carriers.  The  mission  was  to  destroy  the  enemy 
aircraft  parked  on  ramps  at  the  bases  and  to  destroy  aviation 
fuel  stores  at  each  base.   At  the  beginning  of  each  session, 
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the  players  were  given  background  briefing  materials  that 
listed  all  of  the  assets  available  for  the  strike,  the  enemy 
order  of  battle  at  each  of  the  targeted  bases  (intel),  and  the 
desired  probability  of  kill  (Pk)  for  each  target.  Their 
assets  included  five  types  of  aircraft  and  three  types  of 
weapons.  The  briefing  materials  instructed  the  subjects  to 
plan  the  strike  in  two  waves,  with  the  first  wave  hitting  two 
specified  bases,  and  the  second  wave  hitting  the  third  base 
together  with  a  possible  restrike  of  the  lesser  damaged  base 
from  the  first  wave  (ALPHATECH,  Inc.,  1989,  p.  4). 
3.   Hypotheses 

The  major  independent  variables  are  the  presence  of 
the  decision  aid  and  the  difficulty  of  the  planning  task.  The 
dependent  variables  were  divided  into  process  measures  and 
outcome  measures.  The  process  measures  are  situation 
assessment  accuracy,  planning  errors,  information  sought  from 
intel,  time  to  complete  plan,  workload,  and  outcome  prediction 
accuracy.  The  outcome  measure  is  the  success  of  the  air 
strike.  The  goal  of  the  analysis  was  to  understand  the  effect 
of  the  independent  variables  on  outcomes  by  analyzing  their 
effects  on  the  processes  underlying  those  outcomes  (ALPHATECH, 
Inc. ,  1989,  p.  19) . 

The  following  hypotheses  were  tested  in  the 
experiment : 
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•  The  decision  aid  will  reduce  the  time  needed  to  plan  or 
replan  an  air  strike. 

•  Commanders  will  take  advantage  of  the  reduction  in  the 
time  needed  to  plan  to  seek  more  current  information  about 
their  own  or  the  enemy's  situation. 

•  In  the  more  difficult  planning  condition,  time-to-plan 
will  be  longer,  and  the  commander's  subjective  workload 
assessment  will  be  higher. 

•  The  decision  aid  will  reduce  the  commander's  subjective 
workload  during  planning. 

•  Commanders  will  show  more  proactive  information  seeking 
under  conditions  of  lower  subjective  workload. 

•  Proactive  information  seeking  will  lead  to  more  accurate 
enemy  situation  assessment. 

•  More  accurate  enemy  situation  assessment  will  lead  to  more 
accurate  evaluation  of  the  expected  effectiveness  of  the 
plan  (option)  selected.  The  accuracy  of  situation 
assessment  and  effectiveness  evaluation  will  contribute  to 
achievement  of  the  strike  goal. 

•  The  use  of  the  decision  aid  will  result  in  a  plan  with 
greater  internal  consistency  (fewer  errors  made  during 
planning) . 

•  Fewer  planning  errors  will  result  in  better  strike 
outcomes  (achievement  of  strike  goal). 

•  The  effect  of  the  decision  aid  will  be  greater  in  the 
difficult  planning  condition.  (ALPHATECH,  Inc.,  1989,  pp. 
19-20. 


4.   Scenarios 

In  the  experiment,  all  of  the  scenarios  involved  a 
two-wave  air  strike  against  three  enemy  bases.  The  overall 
goal  of  the  strikes  was  to  neutralize  the  air  threat  at  these 
bases.  The  mission  was  the  same  for  all  four  sessions,  and 
was  provided  to  each  player  before  each  session  in  written 
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briefs.  Three  available  Navy  aircraft  carriers  were  located 
approximately  600  miles  from  the  enemy  bases.  The  enemy  bases 
had  air  defenses  in  all  scenarios,  and  Blue  usually  lost  some 
aircraft  in  each  wave.  The  wargame  did  not  support  Blue 
maneuvering  to  avoid  the  enemy  air  defenses,  but  Blue  could 
send  a  SAM  suppression  mission  as  a  part  of  each  strike. 

5.  Amount  of  Intelligence 

Initial  intelligence  information  provided  the  enemy 
order  of  battle  at  each  of  the  targeted  bases.  Intelligence 
reports  gave  battle  damage  assessment  information  within  15 
minutes  after  each  strike  (another  falsehood) .  The  subjects 
could  request  additional  intelligence  reports  from  "national 
sensors"  and  did  get  them.  The  HEAT  observer  noted  the  time 
the  commander  made  the  request  and  instructed  the  RESA 
controller  to  provide  the  information  after  a  delay  of  10-15 
minutes  (another  falsehood) .  The  requested  reports  provided 
additional  battle  damage  assessments  and  estimates  of  enemy 
aircraft  remaining  at  each  base.  The  intelligence  reports 
were  based  on  ground  truth  but  were  not  totally  accurate. 

6.  How  Time  is  Represented 

Each  experiment  sessions  lasted  approximately  three 
hours  and  twenty  minutes.  First,  the  player  was  given  an 
opportunity  to  develop  or  change  an  initial  plan  at  a  time 
rate  of  1-to-l.  The  strike  was  launched  when  the  plan  was 
completed,  and  RESA  was  accelerated  to  run  at  6-to-l  rate. 
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After  the  launch,  the  planes  took  about  30  minutes  (clock 
time)  to  reach  their  targets.  Battle  damage  and  attrition 
information  was  available  in  15  minutes.  The  player  was  given 
an  opportunity  to  replan  based  on  the  new  information,  and 
RESA  was  slowed  to  run  at  1-to-l.  After  the  second  launch, 
the  game  resumed  a  6-to-l  rate,  and  RESA  played  the  air  strike 
to  produce  a  final  set  of  outcome  data. 

C.   PREPARATION\EXECUTION 

1.   Players  and  Control  Personnel 

Four  experienced  air  strike  planners  participated  as 
players.  Two  of  the  players  came  from  the  carrier  Carl  Vinson 
and  had  been  active  in  air  strike  planning.  The  other  two 
players  were  experienced  but  not  currently  assigned  to  air 
strike  planning  duties.  The  following  describes  the  personnel 
set  for  the  decision  aid  condition  followed  by  the  non- 
decision  aid  condition. 

The  Blue  RESA  operator  and  the  Controller/enemy  commander 
were  RESA  lab  staff.  The  N-KRS  operator  was  a  NOSC  staff 
member  involved  in  developing  the  decision  aid.  As 
replanning  occurred,  a  strike  file  was  created  off-line 
for  entry  into  RESA  (this  allowed  an  interface  between  N- 
KRS  and  RESA  to  occur  with  a  negligible  time  delay) ;  the 
operator  handling  this  function  was  provided  by  NOSC.  The 
Blue  player  was  positioned  so  that  he  could  see  the  RESA 
displays  and  talk  with  both  the  RESA  operator  and  the  N- 
KRS  operator.  The  HEAT  observer  was  able  to  record  the 
interactions  that  occurred  and  to  see  the  RESA  and  N-KRS 
displays. 

In  the  non-decision  aid  condition,  the  player  provided 
instructions  for  the  creation  of  the  off-line  RESA  strike 
file.  In  this  condition,  the  player  was  assisted  by  an 
individual  who  did  weaponeering  calculations  to  compute 


60 


probabilities  of  kill,  and  kept  tallies  of  resources 
available   (ALPHATECH,  Inc.,  1989,  p.  15). 


2.   Data  Collection 

Three   types   of   data  were  collected  during   the 
experiment: 

•  data  collected  by  the  HEAT  observer  recording  the 
interaction  of  the  strike  commander  with  the  N-KRS 
operator  in  the  decision  aid  condition  and  the  planning 
assistant  in  the  manual  condition; 

•  questionnaires  completed  by  the  players; 

•  and  strike  outcome  data  from  the  RESA  simulation. 

Data  collected  by  the  HEAT  observer  included: 

•  time  that  any  information  was  received; 

•  all  actions  taken  to  obtain  additional  information, 
including  the  time  of  the  action,  the  type  of  information 
requested  and  received,  and  the  time  of  arrival  of  the 
requested  information; 

•  all  comments  by  the  player  about  the  certainty  of  incoming 
information; 

•  for  each  piece  of  new  information  received,  the  time 
replanning  was  initiated  (if  ever)  in  order  to  adjust  the 
plan  for  new  information,  and  the  time  that  replanning  was 
completed; 

•  the  time  and  nature  of  all  interactions  regarding  changes 
to  be  made  to  the  plan  between  the  player  and  the  N-KRS 
operator  or  the  planning  assistant; 

•  all  options  for  the  air  strike  plan  discussed  by  the 
player; 

•  all  verbal  instructions  given  by  the  player  to  the  RESA 
interface  operator; 
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•  all  comments  by  the  player  about  the  possible  outcomes  of 
any  air  strike  planning  options  being  considered; 

•  all  comments  by  the  player  about  the  plan's  likelihood  of 
success; 

•  all  comments  by  the  player  about  best/worst  case  scenarios 
for  the  plan's  outcomes; 

•  planning  errors  noted  and  corrected  (or  not  corrected), 
including  the  time  the  error  was  noted,  the  type  of  error, 
and  the  time  the  correction  was  initiated  and  completed; 

•  and  all  comments  concerning  the  decision  aid,  including 
ease  or  difficulty  of  use,  features  asked  for  that  were 
not  provided,  and  errors  or  confusion  on  the  players  part 
about  the  capabilities  of  the  decision  aid  (ALPHATECH, 
Inc.,  1989,  p.  17). 


The  players  completed  a  set  of  questionnaires 
following  the  launch  of  each  strike.  The  first  was  a  workload 
questionnaire.  The  second  was  to  assess  situation  assessment 
accuracy  including  estimates  of  the  expected  number  of  targets 
at  each  base  (aircraft)  and  the  confidence  levels  in  those 
estimates.  The  third  questionnaire  assessed  option  evaluation 
accuracy  and  included  estimates  of  the  number  or  percentage  of 
targets  expected  to  be  destroyed  at  each  base  by  the  strike, 
estimates  of  the  number  of  Blue  assets  (planes)  expected  to  be 
lost  and  confidence  levels  for  both  estimates. 

The  data  available  from  RESA  was  the  outcome  of  each 
air  strike  and  on  the  enemy  and  friendly  aircraft  lost  during 
the  strike.  Data  also  included  the  movement  of  enemy  aircraft 
during  the  difficult  scenarios  which  was  used  to  calculate  the 
number  of  enemy  aircraft  remaining  at  the  bases  at  the  time  of 
the  strike. 
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D.   ANALYSIS 

The  following  will  serve  as  examples  of  the  analysis.  The 
report  also  included  analysis  of  the  accuracy  of  estimated 
strike  effectiveness,  planning  errors,  strike  effectiveness 
and  situation  assessment  accuracy. 

1 .   Time  Needed  to  Prepare  Strike  Plan 

One  of  the  primary  hypothesis  is  that  the  decision  aid 

will  save  time  for  the  air  strike  planner.   The  mean  time 

taken  to  prepare  strike  plans  (in  minutes)  with  and  without  N- 

KRS  for  the  easy  and  difficult  experiment  conditions  were 

computed.    The  planning  times  did  not  include  time  spent 

adjusting  available  aircraft  counts  for  aircraft  lost  in 

strikes  or  destroyed  when  carriers  were  attacked.  An  Analysis 

of  Variance  (ANOVA)  was  also  determined  for  the  planning  time. 

Overall,  N-KRS  did  reduce  the  time  needed  to  produce  a 
strike  plan;  in  the  Analysis  of  Variance  (ANOVA)  for 
planning  time,  the  main  effect  of  the  decision  aid  was 
significant  at  the  .10  level.  However,  the  effect  of  the 
decision  aid  on  planning  time  was  concentrated  in  the 
first  wave  of  the  strike.  There  was  a  significant 
difference  in  the  time  taken  to  plan  the  first  wave  of  the 
strike  with  and  without  the  decision  aid  (p<.10).  In 
contrast,  the  time  taken  to  plan  the  second  wave  of  the 
strike  was  similar  with  and  without  the  decision  aid  in 
both  the  easy  and  difficult  conditions.  (ALPHATECH,  Inc., 
1989,  p.  21) 

The  analysis  included  studying  whether  the  players 
allocated  their  planning  time  to  different  activities  when 
using  N-KRS.  The  HEAT  decision  making  process  provided  a 
framework  for  analyzing  the  planning  and  decision  making 
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activities  affected  by  using  N-KRS.   In  the  experiment,  the 

functions  most  relevant  to  air  strike  planning  are  monitoring, 

situation  assessment,  option  generation  and  evaluation.   For 

the  monitoring  function,  the  analysts  studied  the  frequency  of 

requests  for  additional  intelligence  information.  All  of  the 

interactions  between  the  player  and  the  N-KRS  operator  or  the 

planning  assistant  were  recorded  by  the  HEAT  observer.   For 

the  functions  of  situation  assessment  and  option  generation, 

the  recorded  interactions  were  grouped  into  four  decision 

making  activities.   These  activities  were:  enemy  situation 

assessment,   option   exploration,   option   evaluation,   and 

commands  (option  selection) .  A  table  showing  the  distribution 

was  generated  to  show  the  mean  percentages  (the  percentages  of 

interactions  falling  into  each  of  the  four  categories  for  each 

experiment  session  and  these  percentages  were  averaged  across 

subjects  and  across  the  easy/difficult  conditions).    An 

example  of  the  results  was  the  total  number  of  interactions 

observed  in  the  manual  planning  condition  was  much  greater 

(611)  than  in  the  decision  aid  condition  (360),  indicating 

that  more  time  was  spent  planning  in  the  manual  condition. 

2.   Accuracy  of  Enemy  Situation  Assessment 

Before  each  wave  of  the  strike  was  launched,  the 

players  were  asked  to  estimate  how  many  aircraft  they  believed 

would  be  at  the  enemy  bases  when  the  strike  occurred. 

The  accuracy  of  enemy  situation  assessment  was  measured  by 
the  absolute  value  of  deviation  between  the  estimates 
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given  by  the  subjects  and  the  ground  truth  about  the 
number  of  enemy  aircraft  at  each  base,  where  a  smaller 
deviation  indicates  a  more  accurate  estimate. .. .The  mean 
deviation  was  slightly  smaller  without  the  decision  aid  (a 
mean  error  of  38.4  aircraft  without  the  aid  versus  51.8 
aircraft  with  the  aid)  and  the  difference  is  not 
statistically  significant.  The  only  significant 
difference  in  enemy  situation  assessment  accuracy  was 
between  the  easy  and  difficult  conditions.  (ALPHATECH, 
Inc.,  1989,  p.  29) 


E .   REPORT 

1.   Organization 

The  experiment  report  contained  six  chapters  titled: 
Introduction,  Navy  Air  Strike  Planning,  The  N-KRS  Decision 
Aid,  Experiment  Concept  and  Design,  Experiment  Results, 
Implications  for  Improving  N-KRS  and  HEAT  and  Conclusions.  It 
contained  charts,  figures,  tables  and  diagrams.  The  appendix 
included  subject  briefing  materials,  questionnaires  and  data 
collection  forms  and  a  correlation  matrix.  The  Experiment 
Concept  and  Design  chapter  was  further  organized  into  the 
following  subchapters: 

•  Experiment  Overview; 

•  Experiment  Conditions; 

•  Scenarios; 

•  Intel  Information; 

•  Experiment  Set-up  and  Personnel; 

•  Subjects; 

•  and  Data  Collection. 
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2 .   Results 

The  report  overview  of  the  results  is  summarized  as 

follows: 

The  experiment  results  suggest  that  the  strike  planners  in 
the  experiment  were  able  to  develop  equally  effective 
plans  with  and  without  the  N-KRS  aid.  The  major  advantage 
of  the  aid  was  that  it  saved  time  and  prevented 
errors.... As  expected,  the  use  of  the  N-KRS  decision  aid 
was  negatively  related  to  planning  time,  subjective 
workload,  and  planning  time,  subjective  workload,  and 
planning  errors.  Contrary  to  expectation,  it  had  little 
effect  on  information  seeking,  and  little  effect  on  the 
accuracy  of  planner's  assessment  of  the  enemy 
situation. . . .Few  of  the  process  measures  in  the  experiment 
had  any  relationship  to  the  overall  effectiveness  of  the 
strike,  as  measured  by  the  number  of  enemy  aircraft 
destroyed.   (ALPHATECH,  Inc.,  1989,  pp.  37-38) 


The  report  provided  a  figure  which  summarized  the  correlations 

that  were  found  between  the  variables  in  the  experiment.  The 

figure  used  solid  lines  to  indicate  Pearson  correlations  with 

a  significance  level  (two-sided)  of  at  least  p<.15.   Dotted 

lines  were  used  to  show  expected  relationships  that  the  data 

did  not  support.   A  full  correlation  matrix  was  available  in 

the  report  appendix. 

3.   Conclusions 

The  following  paragraph  contains  excerpts  from  the 

report's  summary  of  conclusions. 

Subjects  in  the  experiment  were  able  to  prepare  equally 
feasible  and  effective  air  strike  plans  with  and  without 
the  N-KRS  decision  aid,  but  they  prepared  these  plans  more 
quickly  and  with  fewer  errors  when  N-KRS  was  used.... Use 
of  the  N-KRS  aid  altered  the  process  by  which  the  subjects 
produced  their  strike  plans.... In  interpreting  these 
results,  it  is  important  to  note  that  the  planners  in  the 
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experiment  were  probably  not  yet  as  thoroughly  familiar 
with  the  N-KRS  as  they  would  be  after  weeks  or  months  of 
experience  in  using  it ... .Subjects '  estimates  of  the 
expected  effectiveness  of  the  strike  were  less  accurate 
when  they  used  N-KRS. .. .Subjects  in  the  experiment  often 
commented  that  they  would  have  liked  N-KRS  to  show  then  in 
a  compact,  summary  format  what  weapons  and  aircraft  were 
available  for  the  entire  task  force.  (A1PHATECH,  1989, 
pp.  45-46) 


F.   CRITIQUE 

There  must  be  a  clear  connection  between  the  objective, 
alternative,  and  hypotheses.  The  experiment  design  and 
analytical  plan  continue  this  link.  Finally,  the  conclusions 
should  be  about  the  alternatives  and  be  based  on  the  positive 
or  negative  analysis  results  concerning  the  hypothesis.  The 
experiment  report  needs  to  make  all  these  linkages  apparent. 
The  experiment  report  prepared  by  ALPHATECH,  Inc.  is  a  good 
example.  The  following  questions  can  be  used  effectively  to 
evaluate  C3  experiments.  The  author  has  included  a  response 
pertaining  to  the  N-KRS  experiment. 

The  objectives  need  to  be  clear  and  relevant,  the 
assumptions  reasonable,  and  the  hypotheses  germane. 

•  Was  a  decision  identified  which  requires  answering  now? 
Yes.   The  N-KRS  is  a  working  decision  aid. 

•  Was  the  real  issue  directly  addressed?  Yes.  The  real 
issue  was  how  the  N-KRS  could  best  support  the  strike 
planning  process. 

•  What  question  was  the  analysis  designed  to  answer?  The 
primary  question  was:  Does  the  N-KRS  have  a  measurable 
effect  on  the  air  strike  outcomes?  The  secondary  question 
was:  How  does  the  N-KRS  assist  the  air  strike  planner? 
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•  Could  the  issue  be  settled  by  an  answer  to  the  question? 
Yes  if  both  questions  are  answered  because  the  second 
question  concerns  the  process. 

The  experimental  design  should  find  a  balance  between  being 
comprehensive  and  executable.  The  measures  need  to  be 
discriminatory  and  quantifiable;  and  the  data  replicable. 

•  Were  all  major  factors  effecting  the  question  included? 
Yes,  the  factors  were  the  presence  or  absence  of  the  N-KRS 
decision  aid. 

•  Were  sufficient  levels  of  the  factors  identified?  Yes, 
there  were  two  levels  for  the  difficulty  of  the  scenario: 
easy  and  hard. 

•  Do  the  scenarios  match  the  issue  and  the  question?   Yes. 

•  Is  the  scope  of  the  system  being  considered  adequate?  Yes. 

•  Were  forces  and  weapons  correctly  included?   Yes. 

•  Does  the  operational  concept  reflect  real  doctrine  and 
tactics?  Yes,  except  for  using  the  Pk  factor  in  strike 
planning  to  indicate  target  priority  and  the  intel 
availability. 

•  Are  standard  operating  procedures  followed?  The  report 
did  not  contain  this  information. 

•  Was  the  experiment  adequately  trained  and  staffed?  The 
background  and  experience  level  was  appropriate  for  the 
participants.  The  report  only  says  that  player  training 
in  the  use  of  the  decision  aid  was  limited. 

•  Were  the  supporting  models  matched  in  level  with  the 
issue?   No  information  given. 

•  Were  supporting  models  credible?  The  only  model  described 
in  the  report  was  the  intel  model  and  it  was  not  credible. 

•  Were  high  level  measures  used  (MOFE  and  MOE  rather  than 
MOP)?   No. 

•  Do  the  measures  succinctly  answer  the  question?   Yes. 
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•  Were  both  performance  and  vulnerability  considered? 
Vulnerability  was  not  considered. 

•  Was  a  trial  run  performed?   Information  not  given. 

•  Were  extraneous  factors  like  learning  controlled?  Yes, 
two  players  began  the  experiment  in  the  manual  condition 
and  two  began  by  using  the  decision  aid. 

•  Were  instruments  adequate  to  measure  variables?  Yes  and 
a  variety  of  data  was  collected. 

The  analysis  must  be  correct  and  unbiased.  The 
conclusions  supportable. 

•  Was  the  significance  of  each  factor  established?  Yes. 

•  Did  the  analysis  use  standard  procedures  such  as  ANOVA? 
Yes. 

•  Were  the  conclusions  based  on  the  results?   Yes. 

The  experiment  report  does  not  provide  information 
concerning  RESA,  the  wargame.  The  report  does  not  contain  any 
information  on  the  game  data  base,  the  method  of  gaming 
evaluation,  the  method  for  treating  chance  events  and  RESA 
validation  considerations. 

In  addition  to  evaluating  the  experiment  plan  and 
execution,  the  wargame  itself  needs  to  be  examined. 
Appropriate  questions  for  wargame  analysis  include  the 
following  list  drawn  from  the  appendix  of  The  Art  of  Wargaming 
by  Per la. 

These  questions  concern  player  preparation  for 
participation  in  the  wargame. 
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•  What  information  was  provided  to  participants  prior  to 
their  arrival? 

•  How  are  game  objectives  defined  in  preliminary  briefings? 

•  What  information  is  briefed  to  participants  before  play 
begins? 

•  How  and  to  what  level  of  detail  is  the  scenario  described? 
What  is  it? 


It   is   important   to   have   a   widespread   view   and 
understanding  of  the  structure  and  style  of  the  wargame. 

•  What  is  the  overall  game  structure  and  style? 

•  Who  are  the  players?  Is  there  a  team  structure?  From 
what  commands  to  team  members  and  leaders  come?  What  are 
the  names  and  real-world  jobs  of  the  principal  players? 
How  many  sides  are  there  in  the  game  (one,  two,  or  many)? 
What  are  the  decision  levels  of  the  players,  and  how  do 
they  communicate?  What  are  the  responsibilities  and 
limitations  of  the  players,  and  how  do  these  correspond  to 
their  roles? 

•  What  are  the  roles  of  control?  How  are  command  levels 
above  and  below  the  players  represented?  How  do  players 
and  controllers/umpires  communicate?  What  are 
controller/umpire  responsibilities,  powers,  limitations? 

•  What  is  the  formal  analysis  plan? 

Knowledge  concerning  the  game  execution  is  important  in 
critiquing  a  wargame. 

•  What  data  and  displays  are  available  to  the  players? 

•  During  the  course  of  play,  what  decisions  are  made  by  the 
players,  and  which  are  left  up  to  others  (control, 
umpires)?  Are  players'  questions  focused  on  what  they 
should  do,  what  they  can  do,  what  they  must  do,  what  they 
will  do,  or  how  can  they  do? 

•  How  are  game  events  defined?  What  do  control  and  the 
umpires  not  tell  the  players? 
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How  are  events  sequenced?  What  defines  an  activity?  How 
is  game  time  controlled  relative  to  real  time? 

How  does  battle-damage  assessment  (BDA)  or  event 
resolution  work?  Who  does  BDA?  What  techniques,  models, 
data  do  they  use  and  how? 


As  mentioned  in  Chapter  III,  the  human  element  can  be  a 
benefit  or  weakness  of  the  wargame. 


•  What  are  player's  feelings  about  their  roles  and  ability 
to  influence  events?  What  do  the  players  see  as  the  good 
points  and  the  problems  of  the  process?  What  special 
insights  and  ideas  did  the  players  bring  to  the  game,  and 
how  has  the  play  of  the  game  affected  them? 

•  What  are  the  attitudes  of  the  controllers  and  umpires? 
How  do  they  feel  about  their  role,  and  how  well  are  they 
carrying  it  out?   Is  there  any  source  of  disagreement? 


This  list  is  not  comprehensive  but  will  give  the  reader  a 
foundation  for  forming  questions  unique  to  an  experiment. 
This  chapter  summarizes  a  real  C3  experiment  that  applied 
wargaming  to  test  the  hypotheses  and  uses  the  principles  from 
the  previous  chapters  to  critique  the  example. 
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VI .   SUMMARY 

Naval  military  professionals  need  to  be  able  to 
understand  not  only  what  wargaming  can  do,  but  also  what  it 
cannot  do.  This  thesis  provides  information  about  wargaming 
to  users  who  will  apply  conclusions  derived  from  wargaming 
data  in  decision  making  and  users  of  wargaming  as  a  research 
tool  for  hypotheses  testing.  Game  users  need  to  understand 
and  be  able  to  ask  appropriate  questions  concerning  the 
process  of  using  a  wargame  as  an  experiment  tool. 

The  first  chapter  explains  that  wargame  use  in  general  is 
rising  due  its  ability  to  assist  in  evaluating  combat 
capabilities,  system  improvements  and  new  technology  that  add 
to  the  increasing  complexity  of  warfare.  Wargaming  can 
effectively  be  used  for  identifying  requirements  and 
designing,  developing,  acquiring,  training  for  and  using  major 
C3  systems.  Wargaming  is  particularly  suited  to  exploring  C3 
issues  because  the  game  includes  humans  in  decision  making. 
However,  wargaming  is  not  a  perfect  image  of  reality  and  is  as 
complex  as  the  issues  explored  with  it.  Game  data  users  need 
to  be  aware  of  wargame  limitations  and  hazardous  wargame 
issues.  The  goals  of  this  thesis  were  (1)  to  enable  the 
reader  to  ask  intelligent  questions  concerning  a  wargame 
experiment  before  applying  conclusions  generated  from  wargame 
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data,  and  (2)  to  enable  anyone  desiring  to  use  a  wargame  as  an 
experimental  tool  to  have  a  guide  for  the  experiment  plan. 

The  second  chapter  ensures  that  the  reader  understands 
that  a  wargame  employs  human  beings  as  decision  makers  and  a 
simulation  uses  programmed  rules  to  generate  machine 
decisions.  A  wargame  is  not  real,  not  duplicable  and  not 
useful  for  predicting  future  outcomes.  The  best  use  of  a  game 
is  to  investigate  processes.  Some  general  wargame 
applications  include:  to  explore  not  rigorously  quantifiable 
problems,  to  decompose  complex  problems,  to  uncover  new 
aspects  of  a  problem,  to  test  operational  plans,  to  assist  in 
the  planning  process  and  to  focus  on  human  behavior. 
Applications  exploring  aspects  specific  to  C3  are  command 
roles,  decision  hypothesis  formation,  filtering  and 
interpretation  of  information  and  information  categories  used 
in  making  requests.  Types  of  wargames  are  defined  by: 
technique  choice  for  applying  rules  (rigid  or  free)  and 
opponent  force  knowledge  (open  or  closed),  the  purpose  of  the 
game  (educational,  operational  planning  or  research),  and 
geographic  scope  combined  with  the  level  of  decision  making 
(global/strategic,  theater/operational  or  local/tactical).  A 
list  of  common  features  unique  to  wargames  includes:  a 
wargame  simulates  an  actual  or  predicted  real-life  military 
operation  involving  a  clash  of  opposing  forces;  usually 
represents  at  least  two  forces  and  provides  a  means  for  their 
employment;  follows  accepted  military  rules  and  procedures; 
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and  provides  a  means  for  identifying  and  applying  the 
firepower  of  units  and  weapons.  The  basic  requirements  for 
any  wargame  are  qualified  personnel,  appropriate  facilities 
and  equipment,  a  game  manual  which  provides  the  rules,  data 
and  procedures  to  be  applied  during  the  game,  and  a  scenario 
statement  of  the  situation. 

To  avoid  a  reader  misapplication  of  wargame  data,  the 
nature  of  wargaming,  its  limitations  and  potential  hazards 
were  discussed.  The  conflict  between  balancing  reality  and 
complexity,  and  the  artificialities  such  as  too  much  high 
quality  information  provided  by  communication  circuits  that 
work  too  well,  and  data  validation  problems  contribute  to 
wargame  limitations.  To  facilitate  game  playability,  the 
following  falsehoods  are  acceptable:  assume  that  the  right 
variables  have  been  selected  for  the  experiment,  that  a 
"matched  set"  relationship  exists  between  the  algorithm  and 
data  base  for  the  game,  that  the  study  of  war  can  approximate 
an  exact  science,  that  the  game  uses  current  tactics  and 
doctrine,  and  that  a  wargame  is  repeatable.  Wargaming 
considerations  that  are  hazardous  to  the  accurate  results  of 
game  play  are  poor  documentation,  uncooperative  human  players 
and  misplaced  advocacy. 

Chapter  IV  described  a  broad  outline  of  and  guidelines  for 
designing  a  wargame  experiment.  Experiment  credibility  hinges 
on  the  approach  used  to  plan  the  experiment.  A  clearly 
defined  approach  is  critical  to  ensure  that  the  final  product 
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is  appropriate  for  the  accurate  analysis  of  the  experiment 
results.  The  first  step  of  the  conception  stage  includes  the 
clear  statement  of  the  problem  issues,  objectives  and  response 
variables.  Identifying  the  experiment  alternatives  and 
assumptions,  writing  the  hypothesis,  and  selecting  the 
independent  and  dependent  variables  are  also  included  in  the 
first  step.  The  design  stage  matches  the  experiment 
objectives  with  available  simulation  models,  analysis  tools, 
data  requirements,  measures,  hardware  requirements  and 
security  considerations.  The  detailed  experiment  plan  should 
have  complete  traceability  to  the  originating  problem  and 
issues.  The  design  steps  unique  to  wargaming  are:  composing 
the  scenario  to  allow  player  decision  flexibility,  tailoring 
the  data  base  to  the  game  objectives  and  selecting  the 
software.  When  selecting  the  game,  the  game  objectives,  scope 
and  level,  number  of  sides,  amount  of  intelligence,  methods  of 
evaluation,  representation  of  time,  method  of  treating  chance 
events,  available  models,  game  validation  and  verification, 
and  support  system  requirements  all  need  to  be  evaluated  so 
the  best  possible  matching  of  the  game  capabilities  and 
experiment  requirements  occurs.  Experiment  preparation 
consists  of  establishing  a  schedule,  selecting  players  and 
control  personnel,  creating  forms  and  game  directives  and 
playtesting  to  validate  the  game  system.  During  execution, 
the  game  is  played  and  data  collected.  Analysis  of  the  data 
precedes  writing  the  report. 
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The  last  chapter  applies  the  principles  from  the  previous 
chapters  to  critique  an  example  of  an  actual  C3  experiment. 
It  summarized  the  N-KRS  experiment  report  into  the  stages  of 
conception,  design,  preparation/execution,  analysis  and 
report.  An  appraisal  of  the  example  experiment  design,  based 
on  the  information  contained  in  the  report,  was  provided  by 
listing  evaluation  questions  and  answers.  Very  little 
information  about  the  wargame  was  given  in  the  report  so  only 
questions  for  a  wargame  critique  was  provided. 

The  author  has  provided  a  summary  on  wargaming  and  an 
implementation  guide  for  experimental  wargaming.  Although 
wargaming  is  an  art,  the  reader  is  now  better  prepared  to 
identify  and  avoid  circumstances  of  wargame  abuse  and  data 
misinterpretation . 
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